From berthold at despammed.com Wed Nov 1 15:22:48 2006 From: berthold at despammed.com (Berthold =?iso-8859-15?Q?H=F6llmann?=) Date: Wed, 01 Nov 2006 15:22:48 +0100 Subject: [Distutils] easy_install and Unix python with sys.prefix different from sys.exec_prefix Message-ID: Hello, In our python installation python modules are shared between a Solaris, Linux and Linux-x86_64 installation. We are managing this by giving the --exec-prefix option as well as the --prefix option when configuring python and sharing the directories by NFS. This generally well with distutils, which put extensions under the --exec-prefix path and modules under the --prefix path. The problem with distutils are extensions expecting 'pyconfig.h' at os.path.join(sys.prefix,'install') and shebang string in scripts pointing to a python executable that works only for one architecture. For the shebang string problem it would be helpful to be able to set the default string to #! /usr/bin/env python" in a configuration file. With easy install there is a further problem. It installs extensions into os.path.join(sys.prefix,'lib','python-2.4','site-packages') instead of os.path.join(sys.exec_prefix,'lib','python-2.4','site-packages') which is a problem with the 'easy-install.pth' that point to an architecture sensitive egg in an directory shared by all architectures. Is there a way to fix these problems? Thanks Berthold -- __ Address: G / \ L Germanischer Lloyd phone: +49-40-36149-7374 -+----+- Vorsetzen 35 P.O.Box 111606 fax : +49-40-36149-7320 \__/ D-20459 Hamburg D-20416 Hamburg From pje at telecommunity.com Wed Nov 1 16:47:31 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 01 Nov 2006 10:47:31 -0500 Subject: [Distutils] easy_install and Unix python with sys.prefix different from sys.exec_prefix In-Reply-To: Message-ID: <5.1.1.6.0.20061101103304.02ce8330@sparrow.telecommunity.com> At 03:22 PM 11/1/2006 +0100, Berthold H?llmann wrote: >Is there a way to fix these problems? Set your desired installation paths in one of your configuration files; see the easy_install docs for details. Note, however, that easy_install doesn't have to install to different locations for platform-specific code, because it installs files or directories that include platform information in the filename! So, there's nothing wrong with having a single shared installation directory, as long as you use --multi-version to allow different platform versions to co-exist, without having a default version. The downside to this approach is that if you start the Python interpreter manually instead of via a setuptools-based #! script, then the packages will not be importable without first calling require(). Of course, you can remedy that with a sitecustomize.py containing e.g.: from pkg_resources import require require('somepkg', 'otherpkg', ...) Which would do basically the same thing as having an easy-install.pth, except that it's not platform-specific. The other alternative to this is that you simply use your NFS directory as a cache for easy_install. The way this would work is that you install things with two easy_install steps for each platform: easy_install -zmaxd /path/to/shared/directory SomePackageName easy_install -f /path/to/shared/directory SomePackageName The first line will download/build/etc. and just dump the eggs to the shared directory, without creating a .pth or scripts, etc. Then, the second line will install the .pth and scripts, assuming you've configured easy_install's library and script installation directories to be your platform-specific directories. (Note: you would have to have a platform-specific scripts directory, because the #! line is going to point to the platform-specific Python.) This approach allows you to avoid re-installing platform-generic libraries, because they will be found in the cache directory without downloading. From venkatbo at yahoo.com Thu Nov 2 03:33:09 2006 From: venkatbo at yahoo.com (Venkat Bommakanti) Date: Wed, 1 Nov 2006 18:33:09 -0800 (PST) Subject: [Distutils] PYTHON_EGG_CACHE env setting Message-ID: <20061102023309.33231.qmail@web30506.mail.mud.yahoo.com> Hi, I have some components, that will need to be extracted by the Python eggs runtime (pkg_resources). I understand I can use the PYTHON_EGG_CACHE env setting to specify a desired loaction for extraction. However, was wondering if is there a config file setting or another mechanism to specify this location. Would need to avoid runtime env settings it my deployment. Thanks, /venkat From pje at telecommunity.com Thu Nov 2 05:17:41 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 01 Nov 2006 23:17:41 -0500 Subject: [Distutils] PYTHON_EGG_CACHE env setting In-Reply-To: <20061102023309.33231.qmail@web30506.mail.mud.yahoo.com> Message-ID: <5.1.1.6.0.20061101231457.0282af58@sparrow.telecommunity.com> At 06:33 PM 11/1/2006 -0800, Venkat Bommakanti wrote: >Hi, > > I have some components, that will need to be extracted by > the Python eggs runtime (pkg_resources). I understand I > can use the PYTHON_EGG_CACHE env setting to specify >a desired loaction for extraction. > > >However, was wondering if is there a config file setting or >another mechanism to specify this location. Would need > to avoid runtime env settings it my deployment. See http://peak.telecommunity.com/DevCenter/PkgResources#resource-extraction for the API to change the extraction location, which merely defaults to $PYTHON_EGG_CACHE if not explicitly set. You can use any of the normal Python configuration points (such as sitecustomize.py or a .pth file) to effect this setting. From esj at harvee.org Tue Nov 7 16:42:58 2006 From: esj at harvee.org (Eric S. Johansson) Date: Tue, 07 Nov 2006 10:42:58 -0500 Subject: [Distutils] distutils post processing Message-ID: I'm trying to extend distutiles to do things like fix permissions etc. have a draft that works but I'm not sure how to invoke it. I'm running it after setup() but as you would expect it always run on any setup operation. How can I limit when my code run only when it's a build command? I tried following the distutiles extension instructions but it is a bit thin. some pointers would be appreciated. Thanks for your time. ---eric From victor.stinner at inl.fr Tue Nov 7 16:47:10 2006 From: victor.stinner at inl.fr (Victor Stinner) Date: Tue, 7 Nov 2006 16:47:10 +0100 Subject: [Distutils] setuptools: why does it install ctypes twice? Message-ID: <200611071647.10246.victor.stinner@inl.fr> Hi, I'm one of the authors of pynetfilter_conntrack: http://cheeseshop.python.org/pypi/pynetfilter_conntrack http://software.inl.fr/trac/trac.cgi/wiki/pynetfilter_conntrack Here is the setup.py: http://software.inl.fr/trac/trac.cgi/browser/mirror/edenwall/pynetfilter_conntrack/trunk/setup.py My problem is that "sudo ./setup.py install" reinstall ctypes 1.0.1 and ElementTree 1.2.6 whereas they are already installed by Ubuntu (using apt-...). Should I use different install_requires option? Current one is: install_requires = ("ctypes>=0.9.6", "IPy>=0.42", "ElementTree>=1.0") Bye, Victor Stinner PS: Please CC-me to your answers ;-) -- Victor Stinner http://www.inl.fr/ From pje at telecommunity.com Tue Nov 7 17:39:27 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 07 Nov 2006 11:39:27 -0500 Subject: [Distutils] distutils post processing In-Reply-To: Message-ID: <5.1.1.6.0.20061107113834.01f18790@sparrow.telecommunity.com> At 10:42 AM 11/7/2006 -0500, Eric S. Johansson wrote: >I'm trying to extend distutiles to do things like fix permissions etc. > have a draft that works but I'm not sure how to invoke it. I'm >running it after setup() but as you would expect it always run on any >setup operation. How can I limit when my code run only when it's a >build command? > >I tried following the distutiles extension instructions but it is a bit >thin. some pointers would be appreciated. Thanks for your time. Subclass the appropriate command class, e.g. distuils.command.build_ext.build_ext or whatever, extending the run() method to do what you want. Add it to the 'cmdclass' dictionary in your setup() call. From matt at matt-good.net Tue Nov 7 17:40:56 2006 From: matt at matt-good.net (Matt Good) Date: Tue, 07 Nov 2006 11:40:56 -0500 Subject: [Distutils] setuptools: why does it install ctypes twice? In-Reply-To: <200611071647.10246.victor.stinner@inl.fr> References: <200611071647.10246.victor.stinner@inl.fr> Message-ID: <1162917657.30900.4.camel@nny> On Tue, 2006-11-07 at 16:47 +0100, Victor Stinner wrote: > I'm one of the authors of pynetfilter_conntrack: > http://cheeseshop.python.org/pypi/pynetfilter_conntrack > http://software.inl.fr/trac/trac.cgi/wiki/pynetfilter_conntrack > > Here is the setup.py: > http://software.inl.fr/trac/trac.cgi/browser/mirror/edenwall/pynetfilter_conntrack/trunk/setup.py > > My problem is that "sudo ./setup.py install" reinstall ctypes 1.0.1 and > ElementTree 1.2.6 whereas they are already installed by Ubuntu (using > apt-...). Those Ubuntu packages don't install the metadata ".egg-info" directories needed for setuptools to detect that those modules are already installed. In Ubuntu Edgy there are a few Python packages that do include the .egg-info dirs, but it's still kind of rare. -- Matt Good From pje at telecommunity.com Tue Nov 7 17:49:08 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 07 Nov 2006 11:49:08 -0500 Subject: [Distutils] setuptools: why does it install ctypes twice? In-Reply-To: <200611071647.10246.victor.stinner@inl.fr> Message-ID: <5.1.1.6.0.20061107113936.02b69008@sparrow.telecommunity.com> At 04:47 PM 11/7/2006 +0100, Victor Stinner wrote: >My problem is that "sudo ./setup.py install" reinstall ctypes 1.0.1 and >ElementTree 1.2.6 whereas they are already installed by Ubuntu (using >apt-...). setuptools isn't installing these things twice, it's installing them once. It can't detect that Ubuntu has already installed them, if they are installed without .egg-info files or directories. Python 2.5 does this automatically for any distutils-installed package, but for older versions of Python you need to manually create these files in site-packages if you want to ensure that another copy doesn't get installed: ctypes.egg-info ElementTree.egg-info The content of each file should be that package's PKG-INFO file. This will allow setuptools to detect what versions are installed. The alternative of course is just to let the second installation happen. Still another alternative would be to create a tool for Debian/Ubuntu users that would generate the .egg-info files automatically, although I'm not sure how practical that is, since Debian package names do not map 1:1 with PyPI package names, and the installed packages do not include PKG-INFO files. Essentially, this is a design flaw of Debian packaging policy for Python, which has not been updated to deal with distutils' project-oriented nature or the existence of PyPI. Debian requirements are expressed in terms of Python packages, rather than in terms of Python *projects*. Python projects are the unit of distribution and cataloguing, whereas Python packages are code units, and can be split across projects as well as having more than one in a given project. This problem is independent of setuptools per se, but the flaw is more clearly visible when you try to use setuptools in conjunction with it. From esj at harvee.org Tue Nov 7 21:43:30 2006 From: esj at harvee.org (Eric S. Johansson) Date: Tue, 07 Nov 2006 15:43:30 -0500 Subject: [Distutils] distutils post processing In-Reply-To: <5.1.1.6.0.20061107113834.01f18790@sparrow.telecommunity.com> References: <5.1.1.6.0.20061107113834.01f18790@sparrow.telecommunity.com> Message-ID: <4550EFF2.7000707@harvee.org> Phillip J. Eby wrote: > At 10:42 AM 11/7/2006 -0500, Eric S. Johansson wrote: >> I'm trying to extend distutiles to do things like fix permissions etc. >> have a draft that works but I'm not sure how to invoke it. I'm >> running it after setup() but as you would expect it always run on any >> setup operation. How can I limit when my code run only when it's a >> build command? >> >> I tried following the distutiles extension instructions but it is a bit >> thin. some pointers would be appreciated. Thanks for your time. > > > Subclass the appropriate command class, e.g. > distuils.command.build_ext.build_ext or whatever, extending the run() > method to do what you want. Add it to the 'cmdclass' dictionary in your > setup() call. thank you. Your advice helped by can't quite say why. Somehow it closed a missing gap. I would suggest something more like this inside the documentation for integrating new commands ----------- distutils lets the developer modify different phases of the build, installation, and distribution processes. By overriding one of the distutils/command modules, one can modify the build process for your application. It's often hard to know where to start with regards to modifying setup processes. Frequently it is simpler than it first appears. For example, modifying the mainline build process for pre processing and post processing of data requires a subclass to build and overload the run method. this new run method would do the pre-and or postprocessing necessary. At the right time it would also call the original run method. See the example below. class mybuild(_build): """Specialized Python source builder.""" def run(self): # pre processing # normal processing _build.run(self) # post processing setup would be told about this new class using its cmdclass argument. This argument is a dictionary of class name to look for (i.e. one of the five above) and the class associated with that name. setup(cmdclass={'build': mybuild}, ...) ----- anyway, that's some of the kind of information that would have been useful when I was reading the documentation. From pje at telecommunity.com Tue Nov 7 22:47:55 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 07 Nov 2006 16:47:55 -0500 Subject: [Distutils] distutils post processing In-Reply-To: <4550EFF2.7000707@harvee.org> References: <5.1.1.6.0.20061107113834.01f18790@sparrow.telecommunity.com> <5.1.1.6.0.20061107113834.01f18790@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20061107164738.01f67d00@sparrow.telecommunity.com> At 03:43 PM 11/7/2006 -0500, Eric S. Johansson wrote: >Phillip J. Eby wrote: > > At 10:42 AM 11/7/2006 -0500, Eric S. Johansson wrote: > >> I'm trying to extend distutiles to do things like fix permissions etc. > >> have a draft that works but I'm not sure how to invoke it. I'm > >> running it after setup() but as you would expect it always run on any > >> setup operation. How can I limit when my code run only when it's a > >> build command? > >> > >> I tried following the distutiles extension instructions but it is a bit > >> thin. some pointers would be appreciated. Thanks for your time. > > > > > > Subclass the appropriate command class, e.g. > > distuils.command.build_ext.build_ext or whatever, extending the run() > > method to do what you want. Add it to the 'cmdclass' dictionary in your > > setup() call. > > thank you. Your advice helped by can't quite say why. Somehow it >closed a missing gap. I would suggest something more like this inside >the documentation for integrating new commands http://python.org/doc/2.4.1/dist/node29.html From esj at harvee.org Wed Nov 8 01:02:42 2006 From: esj at harvee.org (Eric S. Johansson) Date: Tue, 07 Nov 2006 19:02:42 -0500 Subject: [Distutils] node29 is evil was: Re: distutils post processing In-Reply-To: <5.1.1.6.0.20061107164738.01f67d00@sparrow.telecommunity.com> References: <5.1.1.6.0.20061107113834.01f18790@sparrow.telecommunity.com> <5.1.1.6.0.20061107113834.01f18790@sparrow.telecommunity.com> <4550EFF2.7000707@harvee.org> <5.1.1.6.0.20061107164738.01f67d00@sparrow.telecommunity.com> Message-ID: Phillip J. Eby wrote: > > http://python.org/doc/2.4.1/dist/node29.html which is why I said: >>>> I tried following the distutiles extension instructions but it is a bit >>>> thin. that node29 documentation has cost me days I will never get back. It is a classic bit of "you will understand this only after you understand it" documentation. to be really useful, this section needs significant expansion. I'm trying to share (by writing) my knowledge as I learn but I'm a bit thin on the ground in this domain. I'll come up with Q's to write about over time and need. --- eric From leif at geodynamics.org Wed Nov 8 04:53:26 2006 From: leif at geodynamics.org (Leif Strand) Date: Tue, 07 Nov 2006 19:53:26 -0800 Subject: [Distutils] Creating an "app folder" with setuptools? Message-ID: <455154B6.8010103@geodynamics.org> I've been trying to use setuptools as a "Pythonic Autopackage". Basically, I want users to be able install a Python package -- plus all its needed dependencies -- with a single command, without any preparatory steps, and with no questions asked. To this end, the best strategy seems to be the "app folder" strategy used on Windows. On Windows, the runtime linker/loader implicitly adds the directory containing the main '.exe' to the DLL search path. This means that regardless of any environment settings (such as PATH), DLLs put alongside the main EXE will be found -- and, with few exceptions, will override DLLs elsewhere on the system. This fact is routinely exploited on Windows: you can dump your EXE and all its needed dependencies in a single folder, and it will "just work". The Python interpreter is similar to the Windows loader in that the directory containing the main Python script is always first on 'sys.path'. This fact, combined with the awesome 'setuptools' package and easy_install's --install-dir (-d) option would seem to give me everything I need. However, there are road blocks which I can't seem to get around (without patching setuptools, that is). The chief problem seems to be that setuptools complains too much! *) It complains if the destination directory doesn't exist. *) It complains if the destination directory doesn't support .pth files or isn't on PYTHONPATH. Actually, I find the errors about .pth/PYTHONPATH to be annoying in general. On Unix, I'm accustomed to tools which simply do what they're told, be it "rm -rf *" or anything else. For example, "./configure --prefix=$HOME/foo/bar; make; make install" will create $HOME/foo/bar if it doesn't already exist, and Autoconf certainly doesn't complain about the subdirectories thus created not being on PATH or PYTHONPATH. One could argue that this amounts to a philosophical difference. But to me, the .pth/PYTHONPATH errors seem like a bug/limitation. If I tell easy_install to create a directory containing my app's scripts, plus all its (possibly "flat") egg dependencies, the resulting directory *will* work once all the dust clears, regardless of what my PYTHONPATH may contain at install time. Even in the case of .pth files, there is no way to know at install time what might work at runtime -- for example, my app's generated, top-level scripts might call site.addsitedir() themselves on behalf of the application. But perhaps I've just overlooked something. Is there a way to to tell setuptools to simply do what it's told? --Leif From pje at telecommunity.com Wed Nov 8 06:54:33 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 08 Nov 2006 00:54:33 -0500 Subject: [Distutils] Creating an "app folder" with setuptools? In-Reply-To: <455154B6.8010103@geodynamics.org> Message-ID: <5.1.1.6.0.20061108005103.01f2b110@sparrow.telecommunity.com> At 07:53 PM 11/7/2006 -0800, Leif Strand wrote: >But perhaps I've just overlooked something. Is there a way to to tell >setuptools to simply do what it's told? -m (--multi-version) tells easy_install that you will be responsible for ensuring the package is on sys.path at runtime. It will not create easy-install.pth (although if there is one in the target directory, it will remove existing references to the package being installed), and it will not check PYTHONPATH or sys.path or .pth support. From nathan at yergler.net Wed Nov 8 13:31:32 2006 From: nathan at yergler.net (Nathan Yergler) Date: Wed, 08 Nov 2006 07:31:32 -0500 Subject: [Distutils] Creating an "app folder" with setuptools? In-Reply-To: <5.1.1.6.0.20061108005103.01f2b110@sparrow.telecommunity.com> References: <5.1.1.6.0.20061108005103.01f2b110@sparrow.telecommunity.com> Message-ID: <1162989092.5197.1.camel@nathan-tablet> On Wed, 2006-11-08 at 00:54 -0500, Phillip J. Eby wrote: > At 07:53 PM 11/7/2006 -0800, Leif Strand wrote: > >But perhaps I've just overlooked something. Is there a way to to tell > >setuptools to simply do what it's told? > > -m (--multi-version) tells easy_install that you will be responsible for > ensuring the package is on sys.path at runtime. It will not create > easy-install.pth (although if there is one in the target directory, it will > remove existing references to the package being installed), and it will not > check PYTHONPATH or sys.path or .pth support. You might also look at zc.buildout; it's zc.recipe.egg recipe has been helpful the times I've used it for installing an egg, along with it's dependencies in a folder outside of site-packages. It also will wrap your scripts with wrappers that add the eggs to the PYTHONPATH. YMMV, of course. Nathan From theller at ctypes.org Wed Nov 8 21:51:36 2006 From: theller at ctypes.org (Thomas Heller) Date: Wed, 08 Nov 2006 21:51:36 +0100 Subject: [Distutils] easy_install and svn Message-ID: <45524358.5050203@ctypes.org> In the easy_install docs is mentioned (at the end): Added subversion download support for svn: and svn+ URLs, as well as automatic recognition of HTTP subversion URLs How does this work? Is it possible to install a package from an svn repository, when the repository resides (read-only) at an http: URL? Thomas From bob at redivi.com Wed Nov 8 22:15:05 2006 From: bob at redivi.com (Bob Ippolito) Date: Wed, 8 Nov 2006 13:15:05 -0800 Subject: [Distutils] easy_install and svn In-Reply-To: <45524358.5050203@ctypes.org> References: <45524358.5050203@ctypes.org> Message-ID: <6a36e7290611081315j68a93639sd563861dd30ee986@mail.gmail.com> On 11/8/06, Thomas Heller wrote: > In the easy_install docs is mentioned (at the end): > > Added subversion download support for svn: and svn+ URLs, as well as automatic recognition of HTTP subversion URLs > > How does this work? Is it possible to install a package from an svn repository, when the repository resides > (read-only) at an http: URL? It works exactly as you'd expect.. $ easy_install http://svn.red-bean.com/bob/simplejson/trunk Downloading http://svn.red-bean.com/bob/simplejson/trunk Doing subversion checkout from http://svn.red-bean.com/bob/simplejson/trunk to /tmp/easy_install-T2h9rD/trunk Processing trunk Running setup.py -q bdist_egg --dist-dir /tmp/easy_install-T2h9rD/trunk/egg-dist-tmp-sjX2PJ Removing simplejson 1.4 from easy-install.pth file Adding simplejson 1.4dev-r977 to easy-install.pth file Installed /Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/site-packages/simplejson-1.4dev_r977-py2.4.egg Processing dependencies for simplejson==1.4dev-r977 -bob From ylddy at motorcitytennis.net Thu Nov 9 09:43:09 2006 From: ylddy at motorcitytennis.net (Helen Mcdonald) Date: Thu, 9 Nov 2006 09:43:09 +0100 Subject: [Distutils] catapult Message-ID: <84D0E1C.D3F394@ptaltd.com> An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20061109/0e128435/attachment.html -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/gif Size: 6184 bytes Desc: not available Url : http://mail.python.org/pipermail/distutils-sig/attachments/20061109/0e128435/attachment.gif From theller at ctypes.org Thu Nov 9 10:21:43 2006 From: theller at ctypes.org (Thomas Heller) Date: Thu, 09 Nov 2006 10:21:43 +0100 Subject: [Distutils] easy_install and svn In-Reply-To: <6a36e7290611081315j68a93639sd563861dd30ee986@mail.gmail.com> References: <45524358.5050203@ctypes.org> <6a36e7290611081315j68a93639sd563861dd30ee986@mail.gmail.com> Message-ID: <4552F327.1070402@ctypes.org> Bob Ippolito schrieb: > On 11/8/06, Thomas Heller wrote: >> In the easy_install docs is mentioned (at the end): >> >> Added subversion download support for svn: and svn+ URLs, as well as automatic recognition of HTTP subversion URLs >> >> How does this work? Is it possible to install a package from an svn repository, when the repository resides >> (read-only) at an http: URL? > > It works exactly as you'd expect.. > > $ easy_install http://svn.red-bean.com/bob/simplejson/trunk > [...] Cool. I had tried something like easy_install -i http://svn.red-bean.com/bob/simplejson/trunk or so, but it didn't work. So, does it make sense to register even unreleased packages with pypi, with the svn url? Thomas From pje at telecommunity.com Thu Nov 9 18:08:15 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 09 Nov 2006 12:08:15 -0500 Subject: [Distutils] easy_install and svn In-Reply-To: <4552F327.1070402@ctypes.org> References: <6a36e7290611081315j68a93639sd563861dd30ee986@mail.gmail.com> <45524358.5050203@ctypes.org> <6a36e7290611081315j68a93639sd563861dd30ee986@mail.gmail.com> Message-ID: <5.1.1.6.0.20061109120424.03fe9b80@sparrow.telecommunity.com> At 10:21 AM 11/9/2006 +0100, Thomas Heller wrote: >Bob Ippolito schrieb: > > On 11/8/06, Thomas Heller wrote: > >> In the easy_install docs is mentioned (at the end): > >> > >> Added subversion download support for svn: and svn+ URLs, as well as > automatic recognition of HTTP subversion URLs > >> > >> How does this work? Is it possible to install a package from an svn > repository, when the repository resides > >> (read-only) at an http: URL? > > > > It works exactly as you'd expect.. > > > > $ easy_install http://svn.red-bean.com/bob/simplejson/trunk > > [...] > >Cool. I had tried something like > easy_install -i http://svn.red-bean.com/bob/simplejson/trunk >or so, but it didn't work. > >So, does it make sense to register even unreleased packages with pypi, >with the >svn url? Yes, but don't put it in the Download URL or Homepage; instead, include the URL in the "long description" with a "#egg=whatever-1.0" tacked on the end. See: http://peak.telecommunity.com/DevCenter/setuptools#making-your-package-available-for-easyinstall for more info. Setuptools itself has two #egg links to subversion in its PyPI page, allowing you to install setuptools==dev (the 0.7 trunk) and setuptools==dev06 (the 0.6 branch) as well as the current "release" version. From theller at ctypes.org Thu Nov 9 18:43:40 2006 From: theller at ctypes.org (Thomas Heller) Date: Thu, 09 Nov 2006 18:43:40 +0100 Subject: [Distutils] list and remove packages with easy_install? Message-ID: <455368CC.7070501@ctypes.org> Ok, easy_install is really cool. Installing most of the packages (*) that one needs with easy_install makes the wish grow to use easy_install as a complete solution. So, why can't easy_install manage the packages that I have installed in these ways: - list the packages that I have installed - remove one of them (completely!) if I don't want to use it any longer. Ok, easy_install -m package removes if from the .pth file, but it is still there on disk. Same if I upgrade a package to a newer version; old versions are left in Lib/site-packages. It looks like all the information that is needed to implement the above two features - is there a reason why this isn't implemented? (*) The one that I rememeber which failed badly was pychecker. IIRC, I ended with batch files in Python24\scripts that contained Python code. But maybe that's pychecker's fault, not easy_install's. From ronaldoussoren at mac.com Thu Nov 9 18:51:59 2006 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Thu, 9 Nov 2006 18:51:59 +0100 Subject: [Distutils] list and remove packages with easy_install? In-Reply-To: <455368CC.7070501@ctypes.org> References: <455368CC.7070501@ctypes.org> Message-ID: <19151DB2-CBFE-4ABA-88C8-A0E98C49D6EC@mac.com> On 9 Nov 2006, at 6:43 PM, Thomas Heller wrote: > Ok, easy_install is really cool. > > Installing most of the packages (*) that one needs with easy_install > makes the wish grow to use easy_install as a complete solution. > > So, why can't easy_install manage the packages that I have > installed in > these ways: > > - list the packages that I have installed > - remove one of them (completely!) if I don't want to use it any > longer. Because nobody got around to building that functionality yet. Feel free to donate this functionality :-) BTW. On my machine: $ packman list altgraph - 0.6.8.dev-r9 - system, develop, active bdist-mpkg - 0.4.3.dev-r14 - system, develop, active cElementTree - 1.0.5-20051216 - system, active Cheetah - 1.0 - system, active CherryPy - 2.2.1 - system, active configobj - 4.3.2 - system, active docutils - 0.4 - system, active ... One of these days I'm going to polish this up and publish it somewhere, it's a command-line wrapper for setuptools. There's also a GUI layer (for OSX only), but that's far from finished. Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3562 bytes Desc: not available Url : http://mail.python.org/pipermail/distutils-sig/attachments/20061109/69ee2a9b/attachment.bin From p.f.moore at gmail.com Thu Nov 9 18:53:03 2006 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 9 Nov 2006 17:53:03 +0000 Subject: [Distutils] list and remove packages with easy_install? In-Reply-To: <455368CC.7070501@ctypes.org> References: <455368CC.7070501@ctypes.org> Message-ID: <79990c6b0611090953k46c4760tc436be0bed1addd5@mail.gmail.com> On 11/9/06, Thomas Heller wrote: > It looks like all the information that is needed to implement the above > two features - is there a reason why this isn't implemented? I believe it's simply a lack of time. Phillip has talked about implementing a "nest" command to manage eggs, but I don't believe he's started writing it yet. A long while ago, I was going to do some work on this, but I never got the time to do so. Paul. From theller at ctypes.org Thu Nov 9 20:38:15 2006 From: theller at ctypes.org (Thomas Heller) Date: Thu, 09 Nov 2006 20:38:15 +0100 Subject: [Distutils] list and remove packages with easy_install? In-Reply-To: <19151DB2-CBFE-4ABA-88C8-A0E98C49D6EC@mac.com> References: <455368CC.7070501@ctypes.org> <19151DB2-CBFE-4ABA-88C8-A0E98C49D6EC@mac.com> Message-ID: <455383A7.1080903@ctypes.org> > On 11/9/06, Thomas Heller wrote: >> It looks like all the information that is needed to implement the above >> two features - is there a reason why this isn't implemented? > Paul Moore schrieb: > I believe it's simply a lack of time. Phillip has talked about > implementing a "nest" command to manage eggs, but I don't believe he's > started writing it yet. A long while ago, I was going to do some work > on this, but I never got the time to do so. Ronald Oussoren schrieb: > Because nobody got around to building that functionality yet. Feel > free to donate this functionality :-) Ok, I'm fine with that answer. Which means that I'll have to wait for it, or have to write my own. I just suspected there were other reasons ;-) Thanks, Thomas From pje at telecommunity.com Thu Nov 9 22:12:58 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 09 Nov 2006 16:12:58 -0500 Subject: [Distutils] list and remove packages with easy_install? In-Reply-To: <455383A7.1080903@ctypes.org> References: <19151DB2-CBFE-4ABA-88C8-A0E98C49D6EC@mac.com> <455368CC.7070501@ctypes.org> <19151DB2-CBFE-4ABA-88C8-A0E98C49D6EC@mac.com> Message-ID: <5.1.1.6.0.20061109161205.01f2d4f8@sparrow.telecommunity.com> At 08:38 PM 11/9/2006 +0100, Thomas Heller wrote: > > On 11/9/06, Thomas Heller wrote: > >> It looks like all the information that is needed to implement the above > >> two features - is there a reason why this isn't implemented? > > > >Paul Moore schrieb: > > I believe it's simply a lack of time. Phillip has talked about > > implementing a "nest" command to manage eggs, but I don't believe he's > > started writing it yet. A long while ago, I was going to do some work > > on this, but I never got the time to do so. > >Ronald Oussoren schrieb: > > Because nobody got around to building that functionality yet. Feel > > free to donate this functionality :-) > >Ok, I'm fine with that answer. Which means that I'll have to wait for it, >or have to write my own. > >I just suspected there were other reasons ;-) Mostly, it amounts to me not having had a big enough for itch those things so as to need to scratch it myself. :) From jorge.vargas at gmail.com Fri Nov 10 02:53:32 2006 From: jorge.vargas at gmail.com (Jorge Vargas) Date: Fri, 10 Nov 2006 01:53:32 +0000 Subject: [Distutils] should I use distutils for a gnome-applet? Message-ID: <32822fe60611091753k1a689064g24d42e59c65ce171@mail.gmail.com> Hi I'm writting a small gnome applet that I'll like to distribute currently I have a small handmade Makefile that will just copy all the files (3) to their destinations. but it depends on gnome-python (gconf,gnomeapplet) and pygtk, so I'll like to have some of the advantages of distutils/setuptools. although I think setuptools here is overkill distutils will be great for automatic packaging. my concern here is that I need to put the files in specific locations outside the python dirs this 2 to be exact. BONOBO_DIR = /usr/lib/bonobo/servers EXE_DIR = /usr/local/bin so my question is will I really win from hoping over to distutils or should I just stick with my Makefile? From steve.ingram at tnei.co.uk Fri Nov 10 13:37:57 2006 From: steve.ingram at tnei.co.uk (Steve Ingram) Date: Fri, 10 Nov 2006 12:37:57 -0000 Subject: [Distutils] Killing Python Interpreter Message-ID: <001001c704c5$0cdf55e0$0f6c5882@SteveI> Hello, I've got a wxPython based dialog application running under windows. No problems when I package this up using py2exe, but when I close the app the process is left running in the background (still listed in task manager). I assume that the dialog closes but the python interpreter is still running, so how do I completely kill the program?? Apologies if this is obvious, too easy to has been answered a 1000 times already, I just can't find the answer!! Thanks Steve -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20061110/47d3bdf5/attachment.html From pje at telecommunity.com Fri Nov 10 18:09:08 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 10 Nov 2006 12:09:08 -0500 Subject: [Distutils] should I use distutils for a gnome-applet? In-Reply-To: <32822fe60611091753k1a689064g24d42e59c65ce171@mail.gmail.co m> Message-ID: <5.1.1.6.0.20061110120818.02c7b0d8@sparrow.telecommunity.com> At 01:53 AM 11/10/2006 +0000, Jorge Vargas wrote: >so my question is will I really win from hoping over to distutils or >should I just stick with my Makefile? Stick with your Makefile and system packaging tools. Distutils is really only a win for Python *libraries*, not applications. From tseaver at palladion.com Fri Nov 10 18:59:44 2006 From: tseaver at palladion.com (Tres Seaver) Date: Fri, 10 Nov 2006 12:59:44 -0500 Subject: [Distutils] should I use distutils for a gnome-applet? In-Reply-To: <5.1.1.6.0.20061110120818.02c7b0d8@sparrow.telecommunity.com> References: <32822fe60611091753k1a689064g24d42e59c65ce171@mail.gmail.co m> <5.1.1.6.0.20061110120818.02c7b0d8@sparrow.telecommunity.com> Message-ID: <4554BE10.20102@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Phillip J. Eby wrote: > At 01:53 AM 11/10/2006 +0000, Jorge Vargas wrote: >> so my question is will I really win from hoping over to distutils or >> should I just stick with my Makefile? > > Stick with your Makefile and system packaging tools. Distutils is really > only a win for Python *libraries*, not applications. zc.buildout has some facilities for automating build of components using "CMMI" (configure + make + make install) in conjunction with setuptools. http://www.zope.org/DevHome/Buildout/ http://svn.zope.org/zc.buildout/trunk/ http://cheeseshop.python.org/pypi/zc.buildout/1.0.0b12 Tres. - -- =================================================================== Tres Seaver +1 202-558-7113 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFFVL4Q+gerLs4ltQ4RAtpKAKCV0yCS0+AUYUQOGMFaLfwIbMAelgCfWZIl s5zExcsmgZCiyPZxzwT4aA0= =S60y -----END PGP SIGNATURE----- From matt at matt-good.net Sat Nov 11 02:03:38 2006 From: matt at matt-good.net (Matt Good) Date: Fri, 10 Nov 2006 20:03:38 -0500 Subject: [Distutils] ORed requirements Message-ID: <1163207018.30900.179.camel@nny> Is there a way to define a requirement like "PackageA or PackageB"? AFAICT there's no syntax to specify this type of requirement, so is there some other way to handle this in setup.py? -- Matt Good From pje at telecommunity.com Sat Nov 11 02:51:28 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 10 Nov 2006 20:51:28 -0500 Subject: [Distutils] ORed requirements In-Reply-To: <1163207018.30900.179.camel@nny> Message-ID: <5.1.1.6.0.20061110204856.01f33bc0@sparrow.telecommunity.com> At 08:03 PM 11/10/2006 -0500, Matt Good wrote: >Is there a way to define a requirement like "PackageA or PackageB"? >AFAICT there's no syntax to specify this type of requirement, so is >there some other way to handle this in setup.py? See http://peak.telecommunity.com/DevCenter/setuptools#declaring-extras-optional-features-with-their-own-dependencies for how to define "extras". You would then tell people to install YourPackage[A] or YourPackage[B] to select which optional dependency they want to use. There is no way to simply choose one to install at random, however. :) From nmichaud at jhu.edu Sat Nov 11 05:13:17 2006 From: nmichaud at jhu.edu (nmichaud at jhu.edu) Date: Fri, 10 Nov 2006 23:13:17 -0500 (EST) Subject: [Distutils] selecting compiler for bdist_rpm Message-ID: Hello, I am trying to compile a module with the Intel compiler, and I can't figure out how to get the bdist_rpm to change the default compiler. I can use build: python setup.py build --compiler=intel but for some reason it doesn't work for bdist_rpm: python setup.py bdist_rpm --compiler=intel gives me error: option --compiler not recognized How can I set the compiler (and CC won't work, because that doesn't set the default linker to use Intel's linker - and then I get all sorts of undefined _intel_memset errors when loading the module). Thanks. Naveen --------------------------------------------------------------------- Naveen Michaud-Agrawal Program in Molecular Biophysics Johns Hopkins University (410) 614 4435 From pje at telecommunity.com Sat Nov 11 07:00:18 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 11 Nov 2006 01:00:18 -0500 Subject: [Distutils] selecting compiler for bdist_rpm In-Reply-To: Message-ID: <5.1.1.6.0.20061111005929.01f45970@sparrow.telecommunity.com> At 11:13 PM 11/10/2006 -0500, nmichaud at jhu.edu wrote: >Hello, > >I am trying to compile a module with the Intel compiler, and I can't >figure out how to get the bdist_rpm to change the default compiler. I can >use build: > >python setup.py build --compiler=intel > >but for some reason it doesn't work for bdist_rpm: > >python setup.py bdist_rpm --compiler=intel > >gives me > >error: option --compiler not recognized > >How can I set the compiler (and CC won't work, because that doesn't set >the default linker to use Intel's linker - and then I get all sorts of >undefined _intel_memset errors when loading the module). Thanks. Put the setting as a default in your setup.cfg: [build] compiler=intel It will then be used by all commands that do builds, including bdist_rpm. From fccoelho at fiocruz.br Sun Nov 12 12:43:38 2006 From: fccoelho at fiocruz.br (Flavio Codeco Coelho) Date: Sun, 12 Nov 2006 09:43:38 -0200 Subject: [Distutils] main script problem Message-ID: <455708EA.4060903@fiocruz.br> Hi, I am having a problem with the fact that setuptools copies the application main script to a different directory during installation. My application is a single package one where the mains script resides within the package. This script imports many of the other modules locally (i.e., import x, y, z) without having to refer to the package. So when the script is copied to /usr/bin, it no longer runs because it cannot import the other modules since they do not reside in the same diretory anymore. This would be a non-issue if setuptools made a simbolic link of the script to /usr/bin instead of copying it. Is there a way of forcing this? any other suggestions are welcome, Fl?vio From pje at telecommunity.com Sun Nov 12 14:05:20 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 12 Nov 2006 08:05:20 -0500 Subject: [Distutils] main script problem In-Reply-To: <455708EA.4060903@fiocruz.br> Message-ID: <5.1.1.6.0.20061112080300.01f32b48@sparrow.telecommunity.com> At 09:43 AM 11/12/2006 -0200, Flavio Codeco Coelho wrote: >Hi, > > I am having a problem with the fact that setuptools copies the >application main script to a different directory during installation. > >My application is a single package one where the mains script resides >within the package. This script imports many of the other modules >locally (i.e., import x, y, z) without having to refer to the package. >So when the script is copied to /usr/bin, it no longer runs because it >cannot import the other modules since they do not reside in the same >diretory anymore. > >This would be a non-issue if setuptools made a simbolic link of the >script to /usr/bin instead of copying it. Is there a way of forcing this? > >any other suggestions are welcome, http://peak.telecommunity.com/DevCenter/setuptools#automatic-script-creation From jorge.vargas at gmail.com Mon Nov 13 11:08:43 2006 From: jorge.vargas at gmail.com (Jorge Vargas) Date: Mon, 13 Nov 2006 10:08:43 +0000 Subject: [Distutils] should I use distutils for a gnome-applet? In-Reply-To: <4554BE10.20102@palladion.com> References: <5.1.1.6.0.20061110120818.02c7b0d8@sparrow.telecommunity.com> <4554BE10.20102@palladion.com> Message-ID: <32822fe60611130208x420a25a5r1c4bcad9e89d6cb9@mail.gmail.com> On 11/10/06, Tres Seaver wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Phillip J. Eby wrote: > > At 01:53 AM 11/10/2006 +0000, Jorge Vargas wrote: > >> so my question is will I really win from hoping over to distutils or > >> should I just stick with my Makefile? > > > > Stick with your Makefile and system packaging tools. Distutils is really > > only a win for Python *libraries*, not applications. umm I have never though of it that way but your right. I just hope my Makefile doesn't grows enough so it needs autotools... > > zc.buildout has some facilities for automating build of components using > "CMMI" (configure + make + make install) in conjunction with setuptools. > I though zc goal was to build zope apps, on the other hand how much of a footprint do you actually need for ac.buildout package? do you need to distribute the whole package bungle with the install? > http://www.zope.org/DevHome/Buildout/ > > http://svn.zope.org/zc.buildout/trunk/ > > http://cheeseshop.python.org/pypi/zc.buildout/1.0.0b12 > > > > Tres. > - -- > =================================================================== > Tres Seaver +1 202-558-7113 tseaver at palladion.com > Palladion Software "Excellence by Design" http://palladion.com > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.2.2 (GNU/Linux) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org > > iD8DBQFFVL4Q+gerLs4ltQ4RAtpKAKCV0yCS0+AUYUQOGMFaLfwIbMAelgCfWZIl > s5zExcsmgZCiyPZxzwT4aA0= > =S60y > -----END PGP SIGNATURE----- > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > From nathan at yergler.net Mon Nov 13 13:58:52 2006 From: nathan at yergler.net (Nathan R. Yergler) Date: Mon, 13 Nov 2006 07:58:52 -0500 Subject: [Distutils] should I use distutils for a gnome-applet? In-Reply-To: <32822fe60611130208x420a25a5r1c4bcad9e89d6cb9@mail.gmail.com> References: <5.1.1.6.0.20061110120818.02c7b0d8@sparrow.telecommunity.com> <4554BE10.20102@palladion.com> <32822fe60611130208x420a25a5r1c4bcad9e89d6cb9@mail.gmail.com> Message-ID: <1163422732.9837.36.camel@Jane> On Mon, 2006-11-13 at 10:08 +0000, Jorge Vargas wrote: [snip] > > > I though zc goal was to build zope apps, on the other hand how much of > a footprint do you actually need for ac.buildout package? do you need > to distribute the whole package bungle with the install? Well I think that Zope Corp's goal for zc.buildout is to build Zope application installations (although I don't speak for them of course), but zc.buildout uses the idea of "recipes" for performing parts of an buildout. And these may or may not involve Zope. For example, I'm using it to deploy a CherryPy-based web app using the provided zc.recipe.egg to install the application egg, its dependencies and exported scripts. Keep in mind that my goal may be a little different than yours -- I want to be able to easily deploy on a server in a self-contained directory without mucking with the system site-packages. The deployment requirements are pretty lightweight -- there's a buildout.cfg (that defines your buildout) and a bootstrap.py that you run to get setuptools, zc.buildout, etc. Read the primary docs for some good stories -- http://cheeseshop.python.org/pypi/zc.buildout/1.0.0b12 . So while I love zc.buildout, if you have a Makefile working, is there something specific you're trying to improve by moving to distutils/setuptools/zc.buildout/etc ? Nathan From jorge.vargas at gmail.com Mon Nov 13 14:40:05 2006 From: jorge.vargas at gmail.com (Jorge Vargas) Date: Mon, 13 Nov 2006 09:40:05 -0400 Subject: [Distutils] should I use distutils for a gnome-applet? In-Reply-To: <1163422732.9837.36.camel@Jane> References: <5.1.1.6.0.20061110120818.02c7b0d8@sparrow.telecommunity.com> <4554BE10.20102@palladion.com> <32822fe60611130208x420a25a5r1c4bcad9e89d6cb9@mail.gmail.com> <1163422732.9837.36.camel@Jane> Message-ID: <32822fe60611130540m33039bc1j13513712f31397d2@mail.gmail.com> On 11/13/06, Nathan R. Yergler wrote: > On Mon, 2006-11-13 at 10:08 +0000, Jorge Vargas wrote: > > [snip] > > > > > > I though zc goal was to build zope apps, on the other hand how much of > > a footprint do you actually need for ac.buildout package? do you need > > to distribute the whole package bungle with the install? > > Well I think that Zope Corp's goal for zc.buildout is to build Zope > application installations (although I don't speak for them of course), > but zc.buildout uses the idea of "recipes" for performing parts of an > buildout. And these may or may not involve Zope. For example, I'm > using it to deploy a CherryPy-based web app using the provided > zc.recipe.egg to install the application egg, its dependencies and > exported scripts. yes I read a bit about it and the recipe seems like a simple ini-based config file. > Keep in mind that my goal may be a little different > than yours -- I want to be able to easily deploy on a server in a > self-contained directory without mucking with the system site-packages. > which is exactly the opposite here gnome apps demand you put certain files (bonobo server fle, gconf schema,etc) at certain places so gnome desktop can load them. > So while I love zc.buildout, if you have a Makefile working, is there > something specific you're trying to improve by moving to > distutils/setuptools/zc.buildout/etc ? in the first place I didn't want to write the Makefile, and since this is python the logical thing will be to use distutils to create python packages, other then that .. nothing. > > > Nathan > > > From jim at zope.com Mon Nov 13 16:02:55 2006 From: jim at zope.com (Jim Fulton) Date: Mon, 13 Nov 2006 10:02:55 -0500 Subject: [Distutils] should I use distutils for a gnome-applet? In-Reply-To: <32822fe60611130208x420a25a5r1c4bcad9e89d6cb9@mail.gmail.com> References: <5.1.1.6.0.20061110120818.02c7b0d8@sparrow.telecommunity.com> <4554BE10.20102@palladion.com> <32822fe60611130208x420a25a5r1c4bcad9e89d6cb9@mail.gmail.com> Message-ID: <4558891F.6050109@zope.com> Jorge Vargas wrote: ... >> zc.buildout has some facilities for automating build of components using >> "CMMI" (configure + make + make install) in conjunction with setuptools. >> > I though zc goal was to build zope apps, zc.buildout really has two goals: 1. Automate system assembly 2. Provide more control over assembly of Python applications. The systems we assemble often include non-python applications, such as ldap servers or spread daemons. For this use case, zc.buildout takes the roll of make accept that it does less. It makes no attempt to do the sort of fine-grained depedency analysis that make does. zc.buildout's main improvement over make is the use of Python as it's scripting language. > on the other hand how much of > a footprint do you actually need for ac.buildout package? do you need > to distribute the whole package bungle with the install? I'm not sure what package you are refering to. To use zc.buildout, you'll need zc.buildout and setuptools, which it depends on. Neither of these are large. BTW, I'm doing experiments with using zc.buildout to generate RPMs for deployment. In this experiment our goal is to install software via RPMs and to perform configuration, including installation of processes such as Zope and database processes dynamically. The software I install includes zc.buildout with enough recipes that I can use zc.buildout to install and update process configurations on demand. Jim -- Jim Fulton mailto:jim at zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From esj at harvee.org Mon Nov 13 22:07:34 2006 From: esj at harvee.org (Eric S. Johansson) Date: Mon, 13 Nov 2006 16:07:34 -0500 Subject: [Distutils] should I use distutils for a gnome-applet? In-Reply-To: <4558891F.6050109@zope.com> References: <5.1.1.6.0.20061110120818.02c7b0d8@sparrow.telecommunity.com> <4554BE10.20102@palladion.com> <32822fe60611130208x420a25a5r1c4bcad9e89d6cb9@mail.gmail.com> <4558891F.6050109@zope.com> Message-ID: Jim Fulton wrote: > zc.buildout's main improvement over > make is the use of Python as it's scripting language. speaking as someone who has detested make from the minute I encountered itsome 30 years ago, I think this is a big improvement. But it also makes me wonder why didn't you use scons? It seems like it does everything make it does only in a more easily understood way and has an established user base. for me the main resistance to using scons is that bumpy case words are significant accessibility impediment. For example, how would you pronounce "Build" and "build"? For most people they sound exactly the same. To differentiate requires a lot of additional noise either spoken or heard. ---eric From jorge.vargas at gmail.com Tue Nov 14 02:10:50 2006 From: jorge.vargas at gmail.com (Jorge Vargas) Date: Tue, 14 Nov 2006 01:10:50 +0000 Subject: [Distutils] zc.buildout vr make Was: should I use distutils for a gnome-applet? Message-ID: <32822fe60611131710t4670a694p65fdb88db1ca52d1@mail.gmail.com> On 11/13/06, Eric S. Johansson wrote: this just got a bit out of topic > Jim Fulton wrote: > > zc.buildout's main improvement over > > make is the use of Python as it's scripting language. > > speaking as someone who has detested make from the minute I encountered > itsome 30 years ago, I think this is a big improvement. same here although I have to say I have much less years alive :) it gets really nasty went you get into the autotools stuff I believe for my needs I'll try to get my hand made Makefile don't get too complicated and don't go agains the flow with the gnome people > But it also > makes me wonder why didn't you use scons? It seems like it does > everything make it does only in a more easily understood way and has an > established user base. > I'm interested in that too, I was considering scons as a alternative for a bigger project of mine, any specific reasons why you left it out > for me the main resistance to using scons is that bumpy case words are > significant accessibility impediment. For example, how would you > pronounce "Build" and "build"? For most people they sound exactly the > same. To differentiate requires a lot of additional noise either spoken > or heard. > > ---eric > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > From bob at redivi.com Tue Nov 14 02:46:53 2006 From: bob at redivi.com (Bob Ippolito) Date: Mon, 13 Nov 2006 17:46:53 -0800 Subject: [Distutils] zc.buildout vr make Was: should I use distutils for a gnome-applet? In-Reply-To: <32822fe60611131710t4670a694p65fdb88db1ca52d1@mail.gmail.com> References: <32822fe60611131710t4670a694p65fdb88db1ca52d1@mail.gmail.com> Message-ID: <6a36e7290611131746j3d521459u30d3f00fc64e7213@mail.gmail.com> On 11/13/06, Jorge Vargas wrote: > On 11/13/06, Eric S. Johansson wrote: > this just got a bit out of topic > > Jim Fulton wrote: > > > zc.buildout's main improvement over > > > make is the use of Python as it's scripting language. > > > > speaking as someone who has detested make from the minute I encountered > > itsome 30 years ago, I think this is a big improvement. > same here although I have to say I have much less years alive :) > it gets really nasty went you get into the autotools stuff > > I believe for my needs I'll try to get my hand made Makefile don't get > too complicated and don't go agains the flow with the gnome people > > But it also > > makes me wonder why didn't you use scons? It seems like it does > > everything make it does only in a more easily understood way and has an > > established user base. > > > I'm interested in that too, I was considering scons as a alternative > for a bigger project of mine, any specific reasons why you left it out > > for me the main resistance to using scons is that bumpy case words are > > significant accessibility impediment. For example, how would you > > pronounce "Build" and "build"? For most people they sound exactly the > > same. To differentiate requires a lot of additional noise either spoken > > or heard. Personally I don't like scons because it doesn't look like Python code to me, and it seems very monolithic. A lot of that is probably because I didn't really start using Python until 2.1.. but I definitely think it could've been done better wrt to catering to Python programmers without making it hard to use for anyone else. I haven't looked at zc.buildout yet, so I can't say if it feels better or worse. -bob From jim at zope.com Tue Nov 14 12:23:47 2006 From: jim at zope.com (Jim Fulton) Date: Tue, 14 Nov 2006 06:23:47 -0500 Subject: [Distutils] should I use distutils for a gnome-applet? In-Reply-To: References: <5.1.1.6.0.20061110120818.02c7b0d8@sparrow.telecommunity.com> <4554BE10.20102@palladion.com> <32822fe60611130208x420a25a5r1c4bcad9e89d6cb9@mail.gmail.com> <4558891F.6050109@zope.com> Message-ID: <4559A743.9020507@zope.com> Eric S. Johansson wrote: > Jim Fulton wrote: >> zc.buildout's main improvement over >> make is the use of Python as it's scripting language. > > speaking as someone who has detested make from the minute I encountered > itsome 30 years ago, I think this is a big improvement. But it also > makes me wonder why didn't you use scons? It seems like it does > everything make it does only in a more easily understood way and has an > established user base. We look ed at scons a couple of years ago and it didn't do what we wanted. I don't remember why. At the time, we were doing make based buildouts. I'll look around and see if I can find out and report back. Jim -- Jim Fulton mailto:jim at zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From pje at telecommunity.com Tue Nov 14 23:16:26 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 14 Nov 2006 17:16:26 -0500 Subject: [Distutils] Sourceforge finally fixed??? Message-ID: <5.1.1.6.0.20061114165945.01f364b8@sparrow.telecommunity.com> I just noticed today that Sourceforge now appears to have a sane download system in place, at long last! Download URLs now point to downloads.sourceforge.net instead of prdownloads, and they automatically select a mirror and redirect to it. If the mirror doesn't have the file, it redirects *back* to downloads.sf.net with a failed_mirror=mirror_name, so that the redirector can pick another mirror. So, in essence, the Sourceforge folks have automated easy_install's mirror selection and retry facility, which means that the code for doing that can possibly be dropped from future versions of easy_install. The new download system appears to be intended to support easier end-user downloads as well as automated downloads, so my guess is that the feature is here to stay. So, I plan to back out my experimental "heroic measures" code (using 'sf-mirrors.telecommunity.com' to get mirror IPs to try), leaving the older prdownloads support in place for the time being. Fortunately, the newer code is only in the 0.6c4 trunk, and not in any officially released version, so the experimental DNS mirroring will not be supported in the 0.6c4 final release, allowing me to drop that (automatically generated) zone from my DNS. Yay! From berthold at despammed.com Wed Nov 15 14:57:50 2006 From: berthold at despammed.com (Berthold =?iso-8859-15?Q?H=F6llmann?=) Date: Wed, 15 Nov 2006 14:57:50 +0100 Subject: [Distutils] easy_install and Unix python with sys.prefix different from sys.exec_prefix References: <5.1.1.6.0.20061101103304.02ce8330@sparrow.telecommunity.com> Message-ID: "Phillip J. Eby" writes: > At 03:22 PM 11/1/2006 +0100, Berthold H?llmann wrote: >>Is there a way to fix these problems? > > Set your desired installation paths in one of your configuration files; see > the easy_install docs for details. > > Note, however, that easy_install doesn't have to install to different > locations for platform-specific code, because it installs files or > directories that include platform information in the filename! So, there's > nothing wrong with having a single shared installation directory, as long > as you use --multi-version to allow different platform versions to > co-exist, without having a default version. > > The downside to this approach is that if you start the Python interpreter > manually instead of via a setuptools-based #! script, then the packages > will not be importable without first calling require(). Of course, you can > remedy that with a sitecustomize.py containing e.g.: > > from pkg_resources import require > require('somepkg', 'otherpkg', ...) > > Which would do basically the same thing as having an easy-install.pth, > except that it's not platform-specific. > > The other alternative to this is that you simply use your NFS directory as > a cache for easy_install. The way this would work is that you install > things with two easy_install steps for each platform: > > easy_install -zmaxd /path/to/shared/directory SomePackageName > easy_install -f /path/to/shared/directory SomePackageName > > The first line will download/build/etc. and just dump the eggs to the > shared directory, without creating a .pth or scripts, etc. Then, the > second line will install the .pth and scripts, assuming you've configured > easy_install's library and script installation directories to be your > platform-specific directories. (Note: you would have to have a > platform-specific scripts directory, because the #! line is going to point > to the platform-specific Python.) > > This approach allows you to avoid re-installing platform-generic libraries, > because they will be found in the cache directory without downloading. After being away from this stuff for a while I want to it. Is there a technical reason for not honoring the difference between "--prefix" and "--exec-prefix" in setuptools the way python and distutils do? With distutils an extensions will be installed into the "exec prefix" path and a module into the "prefix" path. For our purpose I moved the extensions extensions out of the "prefix" site-packages directory into the "exec-prefix" site-packages directory, removed the according lines from the easy-install.pth in the "prefix" tree and copied them to a hand written easy-install.pth in the "exec-prefix" trees. The installation seems to work. I must concede that distutils also have problems with the shebang lines when installing packages. For our own distutils based installations I use from distutils.command import build_scripts build_scripts.first_line_re = re.compile(r'^###!.*python(\s+.*)?$') to fix the shebang line, or better said for not to fix the line. Distutils are looking for the wrong line to replace now. For third party modules I usually fix the shebang lines by hand after the installation from "/bin/python" to "/usr/bin/env python" by hand, but a way of site wide customization would be great. If the problem with implementing my wishes is less technical than lack of available work force, I would be happy to help out, if someone could give me some hint where in the source to look. Thanks Berthold -- __ Address: G / \ L Germanischer Lloyd phone: +49-40-36149-7374 -+----+- Vorsetzen 35 P.O.Box 111606 fax : +49-40-36149-7320 \__/ D-20459 Hamburg D-20416 Hamburg From pje at telecommunity.com Wed Nov 15 17:54:41 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 15 Nov 2006 11:54:41 -0500 Subject: [Distutils] easy_install and Unix python with sys.prefix different from sys.exec_prefix In-Reply-To: References: <5.1.1.6.0.20061101103304.02ce8330@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20061115113843.01f50008@sparrow.telecommunity.com> At 02:57 PM 11/15/2006 +0100, Berthold H?llmann wrote: >After being away from this stuff for a while I want to it. Is there a >technical reason for not honoring the difference between "--prefix" >and "--exec-prefix" in setuptools the way python and distutils do? >With distutils an extensions will be installed into the "exec prefix" >path and a module into the "prefix" path. For our purpose I moved the >extensions extensions out of the "prefix" site-packages directory into >the "exec-prefix" site-packages directory, removed the according lines >from the easy-install.pth in the "prefix" tree and copied them to a >hand written easy-install.pth in the "exec-prefix" trees. The >installation seems to work. When using setuptools, there is no reason to use separate prefix and exec-prefix for library installation, because platform-specific libraries include the platform in their .egg name, so your copying and moving is a waste of time. Just try this (making appropriate substitutions, of course): easy_install -mxd prefix/lib/site-packages ... easy_install -d exec_prefix/lib/site-packages -s exec_prefix/bin ... And repeat both lines for each platform-specific implementation you need, running the appropriate easy_install for the platform. These commands will first install eggs to the $prefix without a .pth or scripts (the -m prevents the .pth, the -x prevents the scripts), and then it will create platform-specific .pth and script files in $exec_prefix. There should be no need for you to manually fudge anything, assuming that both your prefix and exec-prefix's site-packages are on sys.path. Give it a try. From sbllc at elam-binding.com Thu Nov 16 10:50:32 2006 From: sbllc at elam-binding.com (Mercy Mejia) Date: Thu, 16 Nov 2006 09:50:32 -0000 Subject: [Distutils] sardonic Message-ID: <12D72958.9020809@timmxware.com> An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20061116/edb9fb11/attachment.html -------------- next part -------------- A non-text attachment was scrubbed... Name: mastery.gif Type: image/gif Size: 16969 bytes Desc: not available Url : http://mail.python.org/pipermail/distutils-sig/attachments/20061116/edb9fb11/attachment.gif From berthold at despammed.com Thu Nov 16 16:01:43 2006 From: berthold at despammed.com (Berthold =?iso-8859-15?Q?H=F6llmann?=) Date: Thu, 16 Nov 2006 16:01:43 +0100 Subject: [Distutils] easy_install and Unix python with sys.prefix different from sys.exec_prefix References: <5.1.1.6.0.20061101103304.02ce8330@sparrow.telecommunity.com> <5.1.1.6.0.20061115113843.01f50008@sparrow.telecommunity.com> Message-ID: "Phillip J. Eby" writes: > At 02:57 PM 11/15/2006 +0100, Berthold H?llmann wrote: >>After being away from this stuff for a while I want to it. Is there a >>technical reason for not honoring the difference between "--prefix" >>and "--exec-prefix" in setuptools the way python and distutils do? >>With distutils an extensions will be installed into the "exec prefix" >>path and a module into the "prefix" path. For our purpose I moved the >>extensions extensions out of the "prefix" site-packages directory into >>the "exec-prefix" site-packages directory, removed the according lines >>from the easy-install.pth in the "prefix" tree and copied them to a >>hand written easy-install.pth in the "exec-prefix" trees. The >>installation seems to work. > > When using setuptools, there is no reason to use separate prefix and > exec-prefix for library installation, because platform-specific libraries > include the platform in their .egg name, so your copying and moving is a > waste of time. > > Just try this (making appropriate substitutions, of course): > > easy_install -mxd prefix/lib/site-packages ... > easy_install -d exec_prefix/lib/site-packages -s exec_prefix/bin ... > > And repeat both lines for each platform-specific implementation you need, > running the appropriate easy_install for the platform. > > These commands will first install eggs to the $prefix without a .pth or > scripts (the -m prevents the .pth, the -x prevents the scripts), and then > it will create platform-specific .pth and script files in $exec_prefix. > > There should be no need for you to manually fudge anything, assuming that > both your prefix and exec-prefix's site-packages are on sys.path. Give it > a try. I'll certainly give it a try, but why so much hassle for the person installing, when the required information is easily avaliable for the easy_install tool? Hey I read "easy_install" ;-) This still does not solve the problem with the "#!" or shebang lines in scripts. Especially when installing modules, I think using the full path to the python executable is wrong. I'm still pleading for a way to customize the string after the "#!" part. Thank You Berthold -- __ Address: G / \ L Germanischer Lloyd phone: +49-40-36149-7374 -+----+- Vorsetzen 35 P.O.Box 111606 fax : +49-40-36149-7320 \__/ D-20459 Hamburg D-20416 Hamburg From pje at telecommunity.com Thu Nov 16 17:53:22 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 16 Nov 2006 11:53:22 -0500 Subject: [Distutils] easy_install and Unix python with sys.prefix different from sys.exec_prefix In-Reply-To: References: <5.1.1.6.0.20061101103304.02ce8330@sparrow.telecommunity.com> <5.1.1.6.0.20061115113843.01f50008@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20061116114218.01f40ee0@sparrow.telecommunity.com> At 04:01 PM 11/16/2006 +0100, Berthold H?llmann wrote: >I'll certainly give it a try, but why so much hassle for the person >installing, when the required information is easily avaliable for the >easy_install tool? Hey I read "easy_install" ;-) Precisely! Your current directory layout and sharing are intended to work around the fact that in the past, installation of Python packages has *not* been easy for you. If you were starting today, you wouldn't *bother* to do such a complex setup as you're doing, because: It would actually be EASIER to just run easy_install ONCE for each machine or Python installation you needed a package installed for! (It would also be simpler, as there would be no NFS, no split prefix/exec-prefix, etc., etc. At most, you could have an NFS cache directory to save on repeated downloads, as I suggested in a previous post.) So the problem isn't that easy_install is hard, the problem is that you're attempting to make it do something that's really not necessary, because you're accustomed to doing it that way. Nonetheless, I have given you instructions to make it work; you just don't like them. :) >This still does not solve the problem with the "#!" or shebang lines >in scripts. There is no problem, as I understand it. Install scripts to the exec_prefix/bin directories, and they will run the appropriate Python. >Especially when installing modules, I think using the full >path to the python executable is wrong. Ah, there's the problem! Just stop thinking that, and all will be fine. ;) >I'm still pleading for a way to customize the string after the "#!" part. If you really want to spend the effort to do that, I suggest you look at zc.buildout, which extensively wraps setuptools and contains its own #! generation. However it will likely be a lot of work for you to do, to satisfy a concern that is purely a matter of taste rather than functional necessity. (Assuming, again, that I actually understand your requirements correctly.) From berthold at despammed.com Thu Nov 16 18:25:33 2006 From: berthold at despammed.com (Berthold =?iso-8859-15?Q?H=F6llmann?=) Date: Thu, 16 Nov 2006 18:25:33 +0100 Subject: [Distutils] easy_install and Unix python with sys.prefix different from sys.exec_prefix References: <5.1.1.6.0.20061101103304.02ce8330@sparrow.telecommunity.com> <5.1.1.6.0.20061115113843.01f50008@sparrow.telecommunity.com> <5.1.1.6.0.20061116114218.01f40ee0@sparrow.telecommunity.com> Message-ID: "Phillip J. Eby" writes: > At 04:01 PM 11/16/2006 +0100, Berthold H?llmann wrote: >>I'll certainly give it a try, but why so much hassle for the person >>installing, when the required information is easily avaliable for the >>easy_install tool? Hey I read "easy_install" ;-) > > Precisely! Your current directory layout and sharing are intended to work > around the fact that in the past, installation of Python packages has *not* > been easy for you. If you were starting today, you wouldn't *bother* to do > such a complex setup as you're doing, because: > > It would actually be EASIER to just run easy_install ONCE for > each machine or Python installation you needed a package > installed for! Compared to a "wget ", (which is still required for some of the packages I need), untaring the result and "python setup.py install"? Yes it is for extension. But for modules I doubt. An often what I think is a pure python module really contains an extension module and vice versa, which than requires some cleanup. No, it won't really. Our main software is always changing. It consits of a set of quite stable extensions and a big, ever changing pure python module. Installing the whole stuff on three platforms after every change is quite annoying, it is much easier to do the installation of the module on only one platform and benefit from the changes imediately on the others. This is why we choose the actual layout. > (It would also be simpler, as there would be no NFS, no split > prefix/exec-prefix, etc., etc. At most, you could have an NFS cache > directory to save on repeated downloads, as I suggested in a previous post.) I still had to do three installs instead of one for every required module. An not every module or extension allows easy installation now (I need pytables which requires numarray, but easy_install on the latest numarray tar does not install the require header files). > So the problem isn't that easy_install is hard, the problem is that you're > attempting to make it do something that's really not necessary, because > you're accustomed to doing it that way. Nonetheless, I have given you > instructions to make it work; you just don't like them. :) > > >>This still does not solve the problem with the "#!" or shebang lines >>in scripts. > > There is no problem, as I understand it. Install scripts to the > exec_prefix/bin directories, and they will run the appropriate Python. Again three installs for one module instead of one. >>Especially when installing modules, I think using the full >>path to the python executable is wrong. > > Ah, there's the problem! Just stop thinking that, and all will be fine. ;) > > >>I'm still pleading for a way to customize the string after the "#!" part. > > If you really want to spend the effort to do that, I suggest you look at > zc.buildout, which extensively wraps setuptools and contains its own #! > generation. However it will likely be a lot of work for you to do, to > satisfy a concern that is purely a matter of taste rather than functional > necessity. (Assuming, again, that I actually understand your requirements > correctly.) Maybe I could clear our requirements. Regards Berthold -- __ Address: G / \ L Germanischer Lloyd phone: +49-40-36149-7374 -+----+- Vorsetzen 35 P.O.Box 111606 fax : +49-40-36149-7320 \__/ D-20459 Hamburg D-20416 Hamburg From lxander.m at gmail.com Fri Nov 17 18:42:38 2006 From: lxander.m at gmail.com (Alexander Michael) Date: Fri, 17 Nov 2006 12:42:38 -0500 Subject: [Distutils] SetupTools: Running EasyInstall From Within A Script? Message-ID: <525f23e80611170942v6c1f24fby753de97b189cd47c@mail.gmail.com> I am attempting to put together a simple package installation replicator to make it easier to pull updates from a local repository (as an alternative setup to a shared NFS directory). The idea is to put a requirements list and the necessary eggs in a local shared directory and then have a script on each computer do a quick check to see if any updates are needed and install them if there is an update. Is it possible to ask setuptools to install a package from within a Python script without resorting to os.system or friends? I tried browsing the source, but it was difficult to follow the call chain from the easy_install script to the actual method being called. Thanks, Alex From jim at zope.com Fri Nov 17 18:57:11 2006 From: jim at zope.com (Jim Fulton) Date: Fri, 17 Nov 2006 12:57:11 -0500 Subject: [Distutils] SetupTools: Running EasyInstall From Within A Script? In-Reply-To: <525f23e80611170942v6c1f24fby753de97b189cd47c@mail.gmail.com> References: <525f23e80611170942v6c1f24fby753de97b189cd47c@mail.gmail.com> Message-ID: <455DF7F7.2080608@zope.com> Alexander Michael wrote: > I am attempting to put together a simple package installation > replicator to make it easier to pull updates from a local repository > (as an alternative setup to a shared NFS directory). The idea is to > put a requirements list and the necessary eggs in a local shared > directory and then have a script on each computer do a quick check to > see if any updates are needed and install them if there is an update. > Is it possible to ask setuptools to install a package from within a > Python script without resorting to os.system or friends? I tried > browsing the source, but it was difficult to follow the call chain > from the easy_install script to the actual method being called. Technically yes, but not practically. You might look at the easy_install module in zc.buildout, which tries to provide a programmatic interface, although even it resorts to os.spawn at times. See: http://svn.zope.org/zc.buildout/trunk/src/zc/buildout/easy_install.txt?view=markup Jim -- Jim Fulton mailto:jim at zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From pje at telecommunity.com Fri Nov 17 18:59:49 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 17 Nov 2006 12:59:49 -0500 Subject: [Distutils] SetupTools: Running EasyInstall From Within A Script? In-Reply-To: <525f23e80611170942v6c1f24fby753de97b189cd47c@mail.gmail.co m> Message-ID: <5.1.1.6.0.20061117125833.01f52b90@sparrow.telecommunity.com> At 12:42 PM 11/17/2006 -0500, Alexander Michael wrote: >I am attempting to put together a simple package installation >replicator to make it easier to pull updates from a local repository >(as an alternative setup to a shared NFS directory). The idea is to >put a requirements list and the necessary eggs in a local shared >directory and then have a script on each computer do a quick check to >see if any updates are needed and install them if there is an update. >Is it possible to ask setuptools to install a package from within a >Python script without resorting to os.system or friends? I tried >browsing the source, but it was difficult to follow the call chain >from the easy_install script to the actual method being called. The simplest way: from setuptools.command.easy_install import main main(['some', 'arguments', 'here']) From distutils-sig at mlists.thewrittenword.com Fri Nov 17 18:59:06 2006 From: distutils-sig at mlists.thewrittenword.com (Albert Chin) Date: Fri, 17 Nov 2006 11:59:06 -0600 Subject: [Distutils] Why does build_ext accept -R but not build? Message-ID: <20061117175906.GB56906@mail1.thewrittenword.com> Why does "python setup.py build_ext" accept -R but "python setup.py build" does not? Isn't build a superset of build_ext? This is for Python 2.4.2. -- albert chin (china at thewrittenword.com) From pje at telecommunity.com Sat Nov 18 00:04:50 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 17 Nov 2006 18:04:50 -0500 Subject: [Distutils] Why does build_ext accept -R but not build? In-Reply-To: <20061117175906.GB56906@mail1.thewrittenword.com> Message-ID: <5.1.1.6.0.20061117180210.03894ff8@sparrow.telecommunity.com> At 11:59 AM 11/17/2006 -0600, Albert Chin wrote: >Why does "python setup.py build_ext" accept -R but "python setup.py >build" does not? Isn't build a superset of build_ext? No; build just provides defaults for the build_* commands and runs them. Just do "python setup.py build_ext -Rwhatever build"; the build_ext command will only get run once, and with the options you supply. I think you can also do it as "python setup.py build build_ext -Rwhatever", but the commands will run in a slightly different order in that case. From lxander.m at gmail.com Sat Nov 18 18:41:14 2006 From: lxander.m at gmail.com (Alexander Michael) Date: Sat, 18 Nov 2006 12:41:14 -0500 Subject: [Distutils] SetupTools: The Case Of The Installed Egg Message-ID: <525f23e80611180941h163d9b78xedcc824cc1691fc5@mail.gmail.com> In setuptools-0.6c3-py2.4 on win32, installing an egg explicitly results in the installed egg directory having different case than installing it with a version requirement. The following example will explain it more clearly: $ C\:/Python24/Scripts/easy_install dist/Replicator-0.3-py2.4.egg Processing Replicator-0.3-py2.4.egg creating c:\python24\lib\site-packages\Replicator-0.3-py2.4.egg Extracting Replicator-0.3-py2.4.egg to c:\python24\lib\site-packages Adding Replicator 0.3 to easy-install.pth file Installed c:\python24\lib\site-packages\replicator-0.3-py2.4.egg Processing dependencies for Replicator==0.3 Make note of the "creating" message above and compare with the same below. $ C\:/Python24/Scripts/easy_install -f dist Replicator==0.3 Searching for Replicator==0.3 Best match: Replicator 0.3 Processing replicator-0.3-py2.4.egg creating c:\python24\lib\site-packages\replicator-0.3-py2.4.egg Extracting replicator-0.3-py2.4.egg to c:\python24\lib\site-packages Adding replicator 0.3 to easy-install.pth file Installed c:\python24\lib\site-packages\replicator-0.3-py2.4.egg Processing dependencies for Replicator==0.3 It looks like SetupTools now attempts to lower case everything, but doesn't in the case of an explicit install. Bug or feature? Thanks, Alex From nathan at yergler.net Sun Nov 19 18:21:21 2006 From: nathan at yergler.net (Nathan R. Yergler) Date: Sun, 19 Nov 2006 12:21:21 -0500 Subject: [Distutils] Building an egg from Subversion using zc.buildout Message-ID: <1163956881.5763.21.camel@Jane> I'm using zc.buildout for deploying a handful of applications, and one of our dependencies doesn't have eggs or "correctly" formatted pages (that would allow easy_install to track down the repository for direct building from source). Does zc.buildout provide any way to build an egg from a Subversion URL? Something with similar behavior to: easy_install -m -d ./eggs http://path/to/svn/repos I'm happy to write a recipe if necessary, just thought I'd make sure someone else hadn't already scratched this itch. Thanks, Nathan From pje at telecommunity.com Sun Nov 19 19:52:47 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 19 Nov 2006 13:52:47 -0500 Subject: [Distutils] Building an egg from Subversion using zc.buildout In-Reply-To: <1163956881.5763.21.camel@Jane> Message-ID: <5.1.1.6.0.20061119134913.01f3d088@sparrow.telecommunity.com> At 12:21 PM 11/19/2006 -0500, Nathan R. Yergler wrote: >I'm using zc.buildout for deploying a handful of applications, and one >of our dependencies doesn't have eggs or "correctly" formatted pages >(that would allow easy_install to track down the repository for direct >building from source). Does zc.buildout provide any way to build an egg >from a Subversion URL? Something with similar behavior to: > >easy_install -m -d ./eggs http://path/to/svn/repos > >I'm happy to write a recipe if necessary, just thought I'd make sure >someone else hadn't already scratched this itch. If it's a dependency declared in your setup.py, you can do something like this: dependency_links=['http://path/to/svn/repos#egg=Foo-dev'], install_requires=['Foo>=1.2,==dev'], in your setup() call, to depend on version 1.2 or greater of Foo; the ==dev and the #egg=Foo-dev will allow it to be found via SVN. See also these two manual sections that document the use of ==dev and #egg=: http://peak.telecommunity.com/DevCenter/setuptools#managing-continuous-releases-using-subversion http://peak.telecommunity.com/DevCenter/setuptools#making-your-package-available-for-easyinstall From pje at telecommunity.com Sun Nov 19 20:11:41 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 19 Nov 2006 14:11:41 -0500 Subject: [Distutils] SetupTools: The Case Of The Installed Egg In-Reply-To: <525f23e80611180941h163d9b78xedcc824cc1691fc5@mail.gmail.co m> Message-ID: <5.1.1.6.0.20061119140829.027ddc60@sparrow.telecommunity.com> At 12:41 PM 11/18/2006 -0500, Alexander Michael wrote: >It looks like SetupTools now attempts to lower case everything, but >doesn't in the case of an explicit install. Bug or feature? Neither; it's a meaningless quirk caused by the fact that easy_install doesn't case-normalize egg paths supplied via the command line. (Virtually everything else it does is with paths that are "absolutized" and case-normalized.) It doesn't cause any problems, if that's what you're wondering. From lxander.m at gmail.com Mon Nov 20 21:46:22 2006 From: lxander.m at gmail.com (Alexander Michael) Date: Mon, 20 Nov 2006 15:46:22 -0500 Subject: [Distutils] SetupTools: The Case Of The Installed Egg In-Reply-To: <5.1.1.6.0.20061119140829.027ddc60@sparrow.telecommunity.com> References: <5.1.1.6.0.20061119140829.027ddc60@sparrow.telecommunity.com> Message-ID: <525f23e80611201246h2979fca5j88ddade536ac3a98@mail.gmail.com> On 11/19/06, Phillip J. Eby wrote: > At 12:41 PM 11/18/2006 -0500, Alexander Michael wrote: > >It looks like SetupTools now attempts to lower case everything, but > >doesn't in the case of an explicit install. Bug or feature? > > Neither; it's a meaningless quirk caused by the fact that easy_install > doesn't case-normalize egg paths supplied via the command line. (Virtually > everything else it does is with paths that are "absolutized" and > case-normalized.) It doesn't cause any problems, if that's what you're > wondering. Yep. Just wondering if it will bite me later (in linux or cygwin). The thing that concerned me most was that the case of entry inside the easy-install.pth file (all lower) disagreed with what was on the file system (mixed case as specified in setup.py). The different cases resulting from installation method is also a hobgoblin of my little mind, but if it causes no ill, then I can deal with the haunting. Since there is uncertainty in case, is there an easy way with pkg_resources to ask if a particular package is _installed in easy-install.pth_ and meets a given version requirement. For example, something like pkg_resources.isinstalled("MyPackage>=3.2")? Thanks for all your help, Alex From pje at telecommunity.com Mon Nov 20 22:06:18 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 20 Nov 2006 16:06:18 -0500 Subject: [Distutils] SetupTools: The Case Of The Installed Egg In-Reply-To: <525f23e80611201246h2979fca5j88ddade536ac3a98@mail.gmail.co m> References: <5.1.1.6.0.20061119140829.027ddc60@sparrow.telecommunity.com> <5.1.1.6.0.20061119140829.027ddc60@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20061120160331.01f360f0@sparrow.telecommunity.com> At 03:46 PM 11/20/2006 -0500, Alexander Michael wrote: >On 11/19/06, Phillip J. Eby wrote: > > At 12:41 PM 11/18/2006 -0500, Alexander Michael wrote: > > >It looks like SetupTools now attempts to lower case everything, but > > >doesn't in the case of an explicit install. Bug or feature? > > > > Neither; it's a meaningless quirk caused by the fact that easy_install > > doesn't case-normalize egg paths supplied via the command line. (Virtually > > everything else it does is with paths that are "absolutized" and > > case-normalized.) It doesn't cause any problems, if that's what you're > > wondering. > >Yep. Just wondering if it will bite me later (in linux or cygwin). The >thing that concerned me most was that the case of entry inside the >easy-install.pth file (all lower) disagreed with what was on the file >system (mixed case as specified in setup.py). The different cases >resulting from installation method is also a hobgoblin of my little >mind, but if it causes no ill, then I can deal with the haunting. > >Since there is uncertainty in case, is there an easy way with >pkg_resources to ask if a particular package is _installed in >easy-install.pth_ and meets a given version requirement. For example, >something like pkg_resources.isinstalled("MyPackage>=3.2")? pkg_resources.working_set.find('MyPackage>=3.2') will return a true value (a distribution object) if a matching package is currently on sys.path. If 'MyPackage' is on sys.path but doesn't match the version requirement, a VersionConflict error is raised. If it's not on sys.path, None is returned. From Robert.McFadzean at lakelandc.ab.ca Tue Nov 21 02:13:37 2006 From: Robert.McFadzean at lakelandc.ab.ca (Robert McFadzean) Date: Mon, 20 Nov 2006 18:13:37 -0700 Subject: [Distutils] Non-python machines Message-ID: Is it possible to distribute a python module with Python to run on a machine that doesn't have Python or would Python have to be installed separately first? Robert G. McFadzean Instructor Vermilion School of Agriculture Lakeland College 5707 47 Ave West Vermilion AB T9X 1K5 Tlf: (780) 853-4890 Fax: (780) 853-8704 robert.mcfadzean at lakelandc.ab.ca From p.f.moore at gmail.com Tue Nov 21 10:17:35 2006 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 21 Nov 2006 09:17:35 +0000 Subject: [Distutils] Non-python machines In-Reply-To: References: Message-ID: <79990c6b0611210117j1a879fcahefb13950adeecc03@mail.gmail.com> On 11/21/06, Robert McFadzean wrote: > Is it possible to distribute a python module with Python to run on a machine that doesn't have Python or would Python have to be installed separately first? It depends on your platform, but py2exe and cx_Freeze are two options for this. Try google to find the URLs. Paul. From philippe at fluendo.com Tue Nov 21 18:05:03 2006 From: philippe at fluendo.com (Philippe Normand) Date: Tue, 21 Nov 2006 18:05:03 +0100 (CET) Subject: [Distutils] (no subject) Message-ID: <2455.195.10.6.237.1164128703.squirrel@mail.fluendo.com> Hi, When I run that on a fresh checkout of my project (no build/ dir): python setup.py install -O1 --skip-build --root tmp/ I get a nice tmp/usr/lib/python2.4/site-packages/mypackage.egg/ directory (zip_safe set to False in setup()). Then when I run : python setup.py build It creates a build/ directory as expected. Now if I run the first command again, the packages internal to the egg are installed directly in tmp/usr/lib/python2.4/site-packages/ ... Is this a bug or a feature? I reproduce it with 0.6c2 (on Fedora Core 6) and 0.6c3 (on Ubuntu Edgy) versions of setuptools. Philippe From philippe at fluendo.com Tue Nov 21 17:54:53 2006 From: philippe at fluendo.com (Philippe Normand) Date: Tue, 21 Nov 2006 17:54:53 +0100 Subject: [Distutils] [setuptools] setup.py install --skip-build --root "pollutes" site-packages Message-ID: <1164128093.5812.39.camel@localhost.localdomain> Hi, When I run that on a fresh checkout of my project (no build/ dir): python setup.py install -O1 --skip-build --root tmp/ I get a nice tmp/usr/lib/python2.4/site-packages/mypackage.egg/ directory (zip_safe set to False in setup()). Then when I run : python setup.py build It creates a build/ directory as expected. Now if I run the first command again, the packages internal to the egg are installed directly in tmp/usr/lib/python2.4/site-packages/ ... Is this a bug or a feature? I reproduce it with 0.6c2 (on Fedora Core 6) and 0.6c3 (on Ubuntu Edgy) versions of setuptools. Philippe From pje at telecommunity.com Tue Nov 21 19:34:27 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 21 Nov 2006 13:34:27 -0500 Subject: [Distutils] (no subject) In-Reply-To: <2455.195.10.6.237.1164128703.squirrel@mail.fluendo.com> Message-ID: <5.1.1.6.0.20061121124753.040edcd8@sparrow.telecommunity.com> At 06:05 PM 11/21/2006 +0100, Philippe Normand wrote: >Hi, > >When I run that on a fresh checkout of my project (no build/ dir): > >python setup.py install -O1 --skip-build --root tmp/ > >I get a nice tmp/usr/lib/python2.4/site-packages/mypackage.egg/ >directory (zip_safe set to False in setup()). This appears to be a bug. You should not be getting a .egg/ directory here, only a .egg-info/. > Then when I run : > >python setup.py build > >It creates a build/ directory as expected. Now if I run the first >command again, the packages internal to the egg are installed directly >in tmp/usr/lib/python2.4/site-packages/ ... This is the correct behavior when using install --root; the use of --root forces setuptools into a backward-compatible mode to work with system packaging tools such as RPM. >Is this a bug or a feature? I reproduce it with 0.6c2 (on Fedora Core 6) >and 0.6c3 (on Ubuntu Edgy) versions of setuptools. If there's a bug, it would be in the behavior you described for "--skip-build", although I haven't investigated it yet. Could you check and make sure you weren't just seeing an .egg-info directory, rather than an .egg? Thanks. From philippe at fluendo.com Tue Nov 21 20:52:22 2006 From: philippe at fluendo.com (Philippe Normand) Date: Tue, 21 Nov 2006 20:52:22 +0100 (CET) Subject: [Distutils] --skip-build misbehavior In-Reply-To: <5.1.1.6.0.20061121124753.040edcd8@sparrow.telecommunity.com> References: <2455.195.10.6.237.1164128703.squirrel@mail.fluendo.com> <5.1.1.6.0.20061121124753.040edcd8@sparrow.telecommunity.com> Message-ID: <53025.80.102.123.148.1164138742.squirrel@mail.fluendo.com> Phillip J. Eby said: > At 06:05 PM 11/21/2006 +0100, Philippe Normand wrote: >>Hi, >> >>When I run that on a fresh checkout of my project (no build/ dir): >> >>python setup.py install -O1 --skip-build --root tmp/ >> >>I get a nice tmp/usr/lib/python2.4/site-packages/mypackage.egg/ >>directory (zip_safe set to False in setup()). > > This appears to be a bug. You should not be getting a .egg/ directory > here, only a .egg-info/. > > I confirms it's an egg directory, tested again on my home laptop. >> Then when I run : >> >>python setup.py build >> >>It creates a build/ directory as expected. Now if I run the first >> command again, the packages internal to the egg are installed directly >> in tmp/usr/lib/python2.4/site-packages/ ... > > This is the correct behavior when using install --root; the use of > --root forces setuptools into a backward-compatible mode to work with > system packaging tools such as RPM. > > Ah, ok. So I guess I now need to rename my "internal" package as the risk of name clash is too high in this case. I though that by keeping everything in the .egg I would be safe from potential collision. But I was wrong :( Couldn't RPMs (or debs) ship directly the .egg/ directory? >>Is this a bug or a feature? I reproduce it with 0.6c2 (on Fedora Core >> 6) and 0.6c3 (on Ubuntu Edgy) versions of setuptools. > > If there's a bug, it would be in the behavior you described for > "--skip-build", although I haven't investigated it yet. Could you check > and make sure you weren't just seeing an .egg-info directory, rather > than an .egg? Thanks. Well as stated above I double checked on another machine running 0.6c3 on Edgy. BTW sorry for double mail, thought the first one wasn't delivered so used another smtp ;) Philippe From pje at telecommunity.com Wed Nov 22 00:49:20 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 21 Nov 2006 18:49:20 -0500 Subject: [Distutils] --skip-build misbehavior In-Reply-To: <53025.80.102.123.148.1164138742.squirrel@mail.fluendo.com> References: <5.1.1.6.0.20061121124753.040edcd8@sparrow.telecommunity.com> <2455.195.10.6.237.1164128703.squirrel@mail.fluendo.com> <5.1.1.6.0.20061121124753.040edcd8@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20061121184227.01f23140@sparrow.telecommunity.com> At 08:52 PM 11/21/2006 +0100, Philippe Normand wrote: >Ah, ok. So I guess I now need to rename my "internal" package as the risk >of name clash is too high in this case. I though that by keeping >everything in the .egg I would be safe from potential collision. No. Packages are packages. If you install two "foo" modules, they will collide even if they're *inside* of eggs, because Python still only has one internal namespace. Egg separation is primarily to support easy uninstallation, upgrades, and having multiple installed versions (used by different programs). > But I was >wrong :( Couldn't RPMs (or debs) ship directly the .egg/ directory? In principle, yes. In practice, the concept met with extreme resistance in certain quarters, because it requires .pth files to ship with the .egg as well. > >>Is this a bug or a feature? I reproduce it with 0.6c2 (on Fedora Core > >> 6) and 0.6c3 (on Ubuntu Edgy) versions of setuptools. > > > > If there's a bug, it would be in the behavior you described for > > "--skip-build", although I haven't investigated it yet. Could you check > > and make sure you weren't just seeing an .egg-info directory, rather > > than an .egg? Thanks. > >Well as stated above I double checked on another machine running 0.6c3 on >Edgy. Weird. I am not able to reproduce it with 0.6c4dev-r52438 on Linux (Python 2.3, 2.4, or 2.5), nor with 0.7a1 on Windows (Python 2.4) or Cygwin (Python 2.3). In fact, I just plain can't reproduce it. You'll need to give me some more information here. From faassen at infrae.com Wed Nov 22 18:57:07 2006 From: faassen at infrae.com (Martijn Faassen) Date: Wed, 22 Nov 2006 18:57:07 +0100 Subject: [Distutils] buildout and zc.recipe.egg:custom problems Message-ID: <45648F73.7010207@infrae.com> Hi there, I'm trying to induce zc.buildout's zc.recipe.egg:custom recipe to build a library (lxml) against C libraries (libxml2, libxslt) installed in a custom place, i.e. as parts, such as parts/libxml2/include and parts/libxml2/lib. I thought I had that working before and even blogged about it, but it turns out that something's not entirely right yet. Right now I'm struggling to actually see an 'include-dirs' directive to have any effect on what's passed along to the C-compiler (namely the -I) option. It doesn't. After some research I think I've figured out why. I've installed lxml as a development egg, and somehow it always gets built the default way. The custom egg's install code is reached, but only when it's too late and lxml is already built. Therefore any special options such as include-dirs and library-dirs are never passed, as lxml has already been created. I figure lxml gets created automatically and not through the custom recipe as I have it listed as a develop egg in the [buildout] section. What to do about this? Regards, Martijn From jim at zope.com Wed Nov 22 20:31:56 2006 From: jim at zope.com (Jim Fulton) Date: Wed, 22 Nov 2006 14:31:56 -0500 Subject: [Distutils] buildout and zc.recipe.egg:custom problems In-Reply-To: <45648F73.7010207@infrae.com> References: <45648F73.7010207@infrae.com> Message-ID: <4564A5AC.1000706@zope.com> Martijn Faassen wrote: > Hi there, > > I'm trying to induce zc.buildout's zc.recipe.egg:custom recipe to build > a library (lxml) against C libraries (libxml2, libxslt) installed in a > custom place, i.e. as parts, such as parts/libxml2/include and > parts/libxml2/lib. > > I thought I had that working before and even blogged about it, but it > turns out that something's not entirely right yet. > > Right now I'm struggling to actually see an 'include-dirs' directive to > have any effect on what's passed along to the C-compiler (namely the -I) > option. It doesn't. After some research I think I've figured out why. > I've installed lxml as a development egg, and somehow it always gets > built the default way. The custom egg's install code is reached, but > only when it's too late and lxml is already built. Therefore any special > options such as include-dirs and library-dirs are never passed, as lxml > has already been created. > > I figure lxml gets created automatically and not through the custom > recipe as I have it listed as a develop egg in the [buildout] section. > > What to do about this? Unfortunately, you've hit a limitation in buildout and the custom recipe. This limitation will, eventually, be removed. When creating develop eggs, you have no control over build options. When creating custom eggs, you have no way to use directories as source, and besides, you really want a develop egg. I think what we want is a develop recipe that lets you specify various options when creating develop eggs. I'll look at writing this over the next few days. BTW, I think we also need to segregate develop eggs by Python version, since buildout allows creating multiple applications with different Python versions. This is also something I plan in the near future. Jim -- Jim Fulton mailto:jim at zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From philippe at fluendo.com Wed Nov 22 18:04:47 2006 From: philippe at fluendo.com (Philippe Normand) Date: Wed, 22 Nov 2006 18:04:47 +0100 (CET) Subject: [Distutils] --skip-build misbehavior In-Reply-To: <5.1.1.6.0.20061121184227.01f23140@sparrow.telecommunity.com> References: <5.1.1.6.0.20061121124753.040edcd8@sparrow.telecommunity.com> <2455.195.10.6.237.1164128703.squirrel@mail.fluendo.com> <5.1.1.6.0.20061121124753.040edcd8@sparrow.telecommunity.com> <5.1.1.6.0.20061121184227.01f23140@sparrow.telecommunity.com> Message-ID: <4565.195.10.6.237.1164215087.squirrel@mail.fluendo.com> Phillip J. Eby said: > Weird. I am not able to reproduce it with 0.6c4dev-r52438 on Linux > (Python 2.3, 2.4, or 2.5), nor with 0.7a1 on Windows (Python 2.4) or > Cygwin (Python 2.3). In fact, I just plain can't reproduce it. You'll > need to give me some more information here. Oh well, my bad, I was definately all wrong here ;-) It was an egg-info/ ... Next time I'll check more and open my eyes wider before screaming. Philippe From philippe at fluendo.com Wed Nov 22 18:14:17 2006 From: philippe at fluendo.com (Philippe Normand) Date: Wed, 22 Nov 2006 18:14:17 +0100 (CET) Subject: [Distutils] --skip-build misbehavior In-Reply-To: <5.1.1.6.0.20061121184227.01f23140@sparrow.telecommunity.com> References: <5.1.1.6.0.20061121124753.040edcd8@sparrow.telecommunity.com> <2455.195.10.6.237.1164128703.squirrel@mail.fluendo.com> <5.1.1.6.0.20061121124753.040edcd8@sparrow.telecommunity.com> <5.1.1.6.0.20061121184227.01f23140@sparrow.telecommunity.com> Message-ID: <4850.195.10.6.237.1164215657.squirrel@mail.fluendo.com> Phillip J. Eby said: > Weird. I am not able to reproduce it with 0.6c4dev-r52438 on Linux > (Python 2.3, 2.4, or 2.5), nor with 0.7a1 on Windows (Python 2.4) or > Cygwin (Python 2.3). In fact, I just plain can't reproduce it. You'll > need to give me some more information here. Oh well, my bad, I was definately all wrong here ;-) It was an egg-info/ ... Next time I'll check more and open my eyes wider before screaming. Philippe From camargoscomputacao at gmail.com Fri Nov 24 10:55:07 2006 From: camargoscomputacao at gmail.com (Marcel Goncalves Viana Marins de Camargos) Date: Fri, 24 Nov 2006 07:55:07 -0200 Subject: [Distutils] Python and psyco Message-ID: <581b3ffa0611240155w237beb64k9d00d217483eb346@mail.gmail.com> hi, l use python often and i would like to know if you are planning to integrate python with psyco. I need it for speedup. I wasn't able to install psyco on my computer. I use python 2.5. thanks, marcel goncalves -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20061124/89f979d3/attachment.html From fairwinds at eastlink.ca Sun Nov 26 03:57:23 2006 From: fairwinds at eastlink.ca (David Pratt) Date: Sat, 25 Nov 2006 22:57:23 -0400 Subject: [Distutils] Organizing for buildout questions Message-ID: <45690293.8020700@eastlink.ca> Hi. I am trying to get my head around using buildout with both private and public repositories. So my questions relate to a way to organize my svn in a better way for eggification and construction of a simple sandbox to keep my svn checkouts together in a consistent and portable way. The goal is to checkout sources, modifying and checking in my changes as I go, and having the buildouts in their own folders reflect the changing code and dependencies. My first couple of questions are whether buildout can use a folder as source for a (development) egg - something that will update, secondly whether an egg can be created with ssh to a private repository like: svn+ssh://myaccount at myrepo.example.com/usr/home/myaccount/svn/path /to/my/project I have been using ssh for private code sources. In a recent buildout I have been studying, I have seen this: find-links = http://download.zope.org/distribution/ I am also looking at a simple sandbox structure that could also make some sense ... development --+-- private-src (private checked out sources) | +-- public-src (external checked out sources) | +-- buildout (a container for my active buildouts) My source containers would just hold folders of checked out code I am currently working on. I plan on checking out my buildouts from svn to a buildout container so that I could have a few or more different buildouts going at a time (each being an app or just testing some things in their own environment) I am curious about folders as sources since sometimes I just want to try some code without a lot of hassle or committing it to a repository right away. I think it would be good to try it in a buildout. With the structure of my development folder I could use a relative link from a source folder as well. On the repository side, I have been looking at zope's repository more critically which is structured like this: container-->branch-->src-->namespace-->package or when more nesting needed: container-->branch-->src-->namespace-->second namespace-->package setup.py is always at the branch with __init__ for eggs within each namespace package. It seems a good model for eggification since many packages are setup for eggs in a way that is fairly transparent. This of course is a public repository with packages being registered in PyPI. For a private repository, I would like to checkout the code as eggs equally as well. Any recommendations would be appreciated since I am not looking to do this sort of housekeeping any more than once in a long while. I am hoping with input I can get it fairly right and flexible. It seems now is the time to do this to prepare for buildout style development and packaging. Many thanks. Regards, David From antoine.pitrou at wengo.fr Mon Nov 27 14:59:14 2006 From: antoine.pitrou at wengo.fr (Antoine Pitrou) Date: Mon, 27 Nov 2006 14:59:14 +0100 Subject: [Distutils] include files with C extensions Message-ID: <1164635954.5272.15.camel@antoine-ubuntu> Hi, I've got a small problem with the C extension feature. It happens when your C (or Pyrex) extension is spread across several C sources, and you have a custom .h file to make the glue. Say: setup( ... ext_modules = [ Extension('mycext', [ 'src/custom1.c', 'src/custom2.c', ], include_dirs=['src']), ] Let's say custom1.c and custom2.c both include src/custom.h. The problem is when you only change src/custom.h without changing one of the C files. Setuptools does not know that custom.h is part of the C extension code, thus "setup.py build" doesn't rebuild it. Even trying "setup.py clean build" does not result into an actual rebuild. Also, including "src/custom.h" into the extension file list does not work, setuptools complaining that it doesn't know about ".h" files. Is there a clean way to have setuptools do a rebuild when the .h file is changed? Thanks, Antoine. From ronaldoussoren at mac.com Mon Nov 27 15:04:40 2006 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Mon, 27 Nov 2006 06:04:40 -0800 Subject: [Distutils] include files with C extensions In-Reply-To: <1164635954.5272.15.camel@antoine-ubuntu> References: <1164635954.5272.15.camel@antoine-ubuntu> Message-ID: <605F5E25-010F-1000-F221-2596B1B48CC3-Webmail-10006@mac.com> On Monday, November 27, 2006, at 03:01PM, "Antoine Pitrou" wrote: > >Is there a clean way to have setuptools do a rebuild when the .h file is >changed? In recent versions of python (at least 2.4, maybe also 2.3) there's an "depends" argument for Extension where you can list dependencies. Ronald From antoine.pitrou at wengo.fr Mon Nov 27 16:32:46 2006 From: antoine.pitrou at wengo.fr (Antoine Pitrou) Date: Mon, 27 Nov 2006 16:32:46 +0100 Subject: [Distutils] include files with C extensions In-Reply-To: <605F5E25-010F-1000-F221-2596B1B48CC3-Webmail-10006@mac.com> References: <1164635954.5272.15.camel@antoine-ubuntu> <605F5E25-010F-1000-F221-2596B1B48CC3-Webmail-10006@mac.com> Message-ID: <1164641566.5272.19.camel@antoine-ubuntu> Le lundi 27 novembre 2006 ? 06:04 -0800, Ronald Oussoren a ?crit : > >Is there a clean way to have setuptools do a rebuild when the .h file > is > >changed? > > In recent versions of python (at least 2.4, maybe also 2.3) there's an > "depends" argument for Extension where you can list dependencies. Thanks for the tip ! Antoine. From jim at zope.com Tue Nov 28 11:45:17 2006 From: jim at zope.com (Jim Fulton) Date: Tue, 28 Nov 2006 05:45:17 -0500 Subject: [Distutils] Organizing for buildout questions In-Reply-To: <45690293.8020700@eastlink.ca> References: <45690293.8020700@eastlink.ca> Message-ID: <456C133D.9060403@zope.com> David Pratt wrote: > Hi. I am trying to get my head around using buildout with both private > and public repositories. So my questions relate to a way to organize my > svn in a better way for eggification and construction of a simple > sandbox to keep my svn checkouts together in a consistent and portable > way. The goal is to checkout sources, modifying and checking in my > changes as I go, and having the buildouts in their own folders reflect > the changing code and dependencies. > > My first couple of questions are whether buildout can use a folder as > source for a (development) egg - something that will update, Yes. (I wonder if I understand your question.) > secondly > whether an egg can be created with ssh to a private repository like: > > svn+ssh://myaccount at myrepo.example.com/usr/home/myaccount/svn/path > /to/my/project > > I have been using ssh for private code sources. Yes. You check out the projects you want to work on and then configure buildout to create develop eggs from them. The rule of thumb is, use develop eggs for checkouts. It doesn't matter if the checkouts are public or private. Note that you can also have private egg repositories using sftp. See: http://www.python.org/pypi/zc.buildoutsftp > In a recent buildout I have been studying, I have seen this: > > find-links = http://download.zope.org/distribution/ > > I am also looking at a simple sandbox structure that could also make > some sense ... > > development --+-- private-src (private checked out sources) > | > +-- public-src (external checked out sources) > | > +-- buildout (a container for my active buildouts) > > My source containers would just hold folders of checked out code I am > currently working on. I plan on checking out my buildouts from svn to a > buildout container so that I could have a few or more different > buildouts going at a time (each being an app or just testing some things > in their own environment) > > I am curious about folders as sources since sometimes I just want to try > some code without a lot of hassle or committing it to a repository right > away. I think it would be good to try it in a buildout. With the > structure of my development folder I could use a relative link from a > source folder as well. Note that I strongly prefer to keep buildouts self contained. Typically, for each package I work on, I make the svn project for that package a buildout. The buildout.cfg therefore usually has: develop = . I use eggs for everything else needed to work on the package. If, for some reason, I want to work on another package at the same time, I'll check that out into the working directory for the first project(or add an external) and add it to the list of develop eggs. > On the repository side, I have been looking at zope's repository more > critically which is structured like this: > > container-->branch-->src-->namespace-->package Right, where container == project > or when more nesting needed: > > container-->branch-->src-->namespace-->second namespace-->package > > setup.py is always at the branch with __init__ for eggs within each > namespace package. It seems a good model for eggification since many > packages are setup for eggs in a way that is fairly transparent. This of > course is a public repository with packages being registered in PyPI. The main downside of course, is all the nesting, which can get tedious. I wish I could think of a way to have namespace packages without creating all of the annoying subfolders. I may start omitting the src directory in some projects. I also think that it was a mistake to create the zc.recipe namespace packages. Rather than zc.recipe.foo, I should probably have done zc.foorecipe. I may switch to this in the future to reduce the nesting. > For a private repository, I would like to checkout the code as eggs > equally as well. Why not follow the same pattern as for public projects? I don't see why a project's structure should depend on whether it is public or private. Jim -- Jim Fulton mailto:jim at zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From Samuel.Dufour at sophia.inria.fr Tue Nov 28 13:29:17 2006 From: Samuel.Dufour at sophia.inria.fr (Samuel Dufour) Date: Tue, 28 Nov 2006 13:29:17 +0100 Subject: [Distutils] distutils extension and setuptools Message-ID: <1164716957.2930.10.camel@openalea> Hi, We currently develop a software to integrate packages (C++/Python) in plant modeling. We use SCons to build C++ library and Boost.Python to create the wrappers. We have extended distutils to call scons rather than distutils.build_ext. Thus, we distribute packages with the distutils frontend (sdist, bdist_*). All python modules are installed in a common namespace (a meta-package). All libraries and data (shared libraries, headers, tests,...) are installed in a common directory on the system which is external to the python installation (e.g. /usr/local). We would like to use setuptools functionalities to manage dependencies between packages. Is it possible to extend setuptools as we extend distutils to deals correctly with external data ? Thanks. Samuel N.B.: Our own distutils extension (OpenAlea.DistX) provides the following subcommands: - build_scons : call scons with subprocess during the build process. - build_namespace : create meta packages (just a directory and a __init__.py). - install_external_data : copy some data in a custom place. We adapt sdist and bdist_* commands to manage this kind of data. It is like the 'data_file' field, but add recursive copy and the detection of base destination directory depending of the system configuration. - set_env_var : set windows environment variable to discover shared libs at runtime. OpenAlea.DistX is available on http://gforge.inria.fr/frs/?group_id=79&release_id=782 From jim at zope.com Tue Nov 28 19:32:39 2006 From: jim at zope.com (Jim Fulton) Date: Tue, 28 Nov 2006 13:32:39 -0500 Subject: [Distutils] buildout WHUI heads up Message-ID: <456C80C7.4050400@zope.com> This is a heads up that I'm going to call "we haven't used it!" on a couple of buildout features. The next release of buildout will not support ConfigParser-style variable substitutions (e.g. %(foo)s). Only the string-template style of variable (e.g. ${section:option}) substitutions will be supported. Supporting both violates "there's only one way to do it". I'm deprecating the buildout-section extendedBy option. Ian suggested to me a while ago that this wasn't needed and I've come to agree with him. If anyone is using these features and will have pain by having them disappear, please let me know. It will be at least a couple of days, probably more, before I make a new release. Jim -- Jim Fulton mailto:jim at zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From pje at telecommunity.com Tue Nov 28 20:16:14 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 28 Nov 2006 14:16:14 -0500 Subject: [Distutils] distutils extension and setuptools In-Reply-To: <1164716957.2930.10.camel@openalea> Message-ID: <5.1.1.6.0.20061128140619.02882700@sparrow.telecommunity.com> At 01:29 PM 11/28/2006 +0100, Samuel Dufour wrote: >Hi, > >We currently develop a software to integrate packages (C++/Python) in >plant modeling. We use SCons to build C++ library and Boost.Python to >create the wrappers. We have extended distutils to call scons rather >than distutils.build_ext. Thus, we distribute packages with the >distutils frontend (sdist, bdist_*). > >All python modules are installed in a common namespace (a meta-package). >All libraries and data (shared libraries, headers, tests,...) are >installed in a common directory on the system which is >external to the python installation (e.g. /usr/local). > >We would like to use setuptools functionalities to manage dependencies >between packages. Is it possible to extend setuptools as we extend >distutils to deals correctly with external data ? Setuptools doesn't support installation outside of a single location for libraries and a single location for scripts, and extending it to do so isn't really possible if you want your project to be easy_install-able (i.e. support dependencies or be depended upon). At best, you could use --single-version-externally-managed to install a project with external data, but that option isn't compatible with easy_install. From Robert.McFadzean at lakelandc.ab.ca Thu Nov 30 01:28:56 2006 From: Robert.McFadzean at lakelandc.ab.ca (Robert McFadzean) Date: Wed, 29 Nov 2006 17:28:56 -0700 Subject: [Distutils] Specifying a compiler Message-ID: The documentation indicates that one can specify a compiler as in build --compiler=vc98. I have Visual Studio VC98 C++ 6.0 vc98 doesn't work to specify the compiler. Does anyone know how I would specify it so setup will use what I have, rather than looking for VC7.0 that was used to compile python? Robert G. McFadzean Instructor Vermilion School of Agriculture Lakeland College 5707 47 Ave West Vermilion AB T9X 1K5 Tlf: (780) 853-4890 Fax: (780) 853-8704 robert.mcfadzean at lakelandc.ab.ca From venkatbo at yahoo.com Thu Nov 30 01:48:59 2006 From: venkatbo at yahoo.com (Venkat Bommakanti) Date: Wed, 29 Nov 2006 16:48:59 -0800 (PST) Subject: [Distutils] Specifying a compiler In-Reply-To: Message-ID: <299614.42504.qm@web30505.mail.mud.yahoo.com> --- Robert McFadzean wrote: > The documentation indicates that one can specify a > compiler as in build --compiler=vc98. I have Visual > Studio VC98 C++ 6.0 vc98 doesn't work to specify > the compiler. Does anyone know how I would specify > it so setup will use what I have, rather than > looking for VC7.0 that was used to compile python? > Hi Robert, I think there is a fixed set of "supported compilers" identified by specific "names"... I think /Lib/distutils/ccompiler.py lists those pre-determined "named compilers". You may need to use "--compiler=msvc" and ensure your compiler of choice is aliased to "msvc" (probably what setuptools is expecting) Hope this helps... unsure tho... /venkat > Robert G. McFadzean > Instructor > Vermilion School of Agriculture > Lakeland College > 5707 47 Ave West > Vermilion AB T9X 1K5 > Tlf: (780) 853-4890 > Fax: (780) 853-8704 > robert.mcfadzean at lakelandc.ab.ca > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > ____________________________________________________________________________________ Cheap talk? Check out Yahoo! Messenger's low PC-to-Phone call rates. http://voice.yahoo.com From pje at telecommunity.com Thu Nov 30 05:28:37 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 29 Nov 2006 23:28:37 -0500 Subject: [Distutils] Specifying a compiler In-Reply-To: Message-ID: <5.1.1.6.0.20061129232222.028b8cb0@sparrow.telecommunity.com> At 05:28 PM 11/29/2006 -0700, Robert McFadzean wrote: >The documentation indicates that one can specify a compiler as in build >--compiler=vc98. Actually, you have to use one of these: $ python setup.py build --help-compiler List of available compilers: --compiler=bcpp Borland C++ Compiler --compiler=cygwin Cygwin port of GNU C Compiler for Win32 --compiler=emx EMX port of GNU C Compiler for OS/2 --compiler=mingw32 Mingw32 port of GNU C Compiler for Win32 --compiler=msvc Microsoft Visual C++ --compiler=mwerks MetroWerks CodeWarrior --compiler=unix standard UNIX-style compiler >I have Visual Studio VC98 C++ 6.0 vc98 doesn't work to specify the >compiler. Does anyone know how I would specify it so setup will use what >I have, rather than looking for VC7.0 that was used to compile python? I suspect the answer is that you can't, and have it still work. You probably have to download one of the "free" MS compilers. There may also be some sort of environment variable and/or registry hacking that can be done to force Python to use your older compiler, but it's not clear that the resulting extensions will actually work with your Python. Alternately, you can use the free MinGW compiler. See: http://www.mingw.org/MinGWiki/index.php/Python%20extensions for one possible source of instructions. From lxander.m at gmail.com Thu Nov 30 17:47:07 2006 From: lxander.m at gmail.com (Alexander Michael) Date: Thu, 30 Nov 2006 11:47:07 -0500 Subject: [Distutils] SetupTools: The Case Of The Installed Egg In-Reply-To: <5.1.1.6.0.20061120160331.01f360f0@sparrow.telecommunity.com> References: <5.1.1.6.0.20061119140829.027ddc60@sparrow.telecommunity.com> <5.1.1.6.0.20061120160331.01f360f0@sparrow.telecommunity.com> Message-ID: <525f23e80611300847m5d344cfw31826d100ec4d723@mail.gmail.com> On 11/20/06, Phillip J. Eby wrote: > At 03:46 PM 11/20/2006 -0500, Alexander Michael wrote: > >On 11/19/06, Phillip J. Eby wrote: > > > At 12:41 PM 11/18/2006 -0500, Alexander Michael wrote: > > > >It looks like SetupTools now attempts to lower case everything, but > > > >doesn't in the case of an explicit install. Bug or feature? > > > > > > Neither; it's a meaningless quirk caused by the fact that easy_install > > > doesn't case-normalize egg paths supplied via the command line. (Virtually > > > everything else it does is with paths that are "absolutized" and > > > case-normalized.) It doesn't cause any problems, if that's what you're > > > wondering. > > > >Yep. Just wondering if it will bite me later (in linux or cygwin). The > >thing that concerned me most was that the case of entry inside the > >easy-install.pth file (all lower) disagreed with what was on the file > >system (mixed case as specified in setup.py). The different cases > >resulting from installation method is also a hobgoblin of my little > >mind, but if it causes no ill, then I can deal with the haunting. > > > >Since there is uncertainty in case, is there an easy way with > >pkg_resources to ask if a particular package is _installed in > >easy-install.pth_ and meets a given version requirement. For example, > >something like pkg_resources.isinstalled("MyPackage>=3.2")? > > pkg_resources.working_set.find('MyPackage>=3.2') will return a true value > (a distribution object) if a matching package is currently on sys.path. If > 'MyPackage' is on sys.path but doesn't match the version requirement, a > VersionConflict error is raised. If it's not on sys.path, None is returned. Just got back around to trying this...looks like working_set.find wants a Requirement object. The following seemed to work: pkg_resources.working_set.find(pkg_resources.Requirement.parse('MyPackage>=3.2')) Thanks! From S.Pascoe at rl.ac.uk Wed Nov 29 10:42:21 2006 From: S.Pascoe at rl.ac.uk (Pascoe, S (Stephen)) Date: Wed, 29 Nov 2006 09:42:21 -0000 Subject: [Distutils] parse_version() with '-' Message-ID: I've just noticed pkg_resources.parse_version() doesn't appear to follow the setuptools documentation with regards 'rc' vs. '-rc'. With setuptools-0.6c3: Python 2.4.1 (#1, Oct 13 2006, 16:58:04) [GCC 4.0.2 20050901 (prerelease) (SUSE Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> from pkg_resources import parse_version >>> parse_version('2.1-rc2') < parse_version('2.1') True >>> parse_version('2.1rc2') < parse_version('2.1') True >>> This contradicts the docs and suggests a '-' no longer differentiates pre and post releases. Is this a bug or feature? Thanks, Stephen. --- Stephen Pascoe 01235 445980 British Atmospheric Data Centre Rutherford Appleton Laboratory, CCLRC -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20061129/aa57dc97/attachment.htm