From pje at telecommunity.com Sun Oct 1 23:01:30 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 01 Oct 2006 17:01:30 -0400 Subject: [Distutils] setuptools and data files In-Reply-To: <451EC8F5.2080005@livinglogic.de> Message-ID: <5.1.1.6.0.20061001165435.02770e08@unrestrained.com> At 09:43 PM 9/30/2006 +0200, Walter D?rwald wrote: >Hello Philip! > >I'm having trouble with using setuptools with data files. The layout of >my directory is this: > >./src >./src/CVS >./src/CVS/Root >./src/CVS/Repository >./src/CVS/Entries >./src/coverage.css >./src/pycoco.py >./CVS >./CVS/Root >./CVS/Repository >./CVS/Entries >./setup.cfg >./installer.bmp >./MANIFEST.in >./setup.py > >MANIFEST.in contains the line: >include src/coverage.css Setuptools only includes data files that are inside a *package* (which is why it's called "package_data"), or inside the .egg-info directory (metadata files), as that's the only way to ensure the files are accessible in all possible installation configurations (egg zips, egg dirs, and "traditional" installs). >My arguments to setup() look like this: > > args = dict( > name="pycoco", > version="0.1", > description="Python code coverage", > long_description="...", > author=u"Walter Doerwald", > author_email="walter at livinglogic.de", > url="...", > download_url="...", > license="Python", > classifiers="...", > keywords="...", > package_dir={"": "src"}, > py_modules=["pycoco"], > package_data={ > "": ["*.css"], > } > ) For this configuration, I would suggest that you move the .css file to the package's .egg-info directory, and use this expression at runtime to retrieve the contents: data = pkg_resources.get_distribution('pycoco').get_metadata('coverage.css') >However when I do an "easy_install -Z ." I get the following directory >layout: > >/var/home/coverage/pythonroot/pycoco-0.1-py2.5.egg/ >/var/home/coverage/pythonroot/pycoco-0.1-py2.5.egg/pycoco.py >/var/home/coverage/pythonroot/pycoco-0.1-py2.5.egg/pycoco.pyc >/var/home/coverage/pythonroot/pycoco-0.1-py2.5.egg/EGG-INFO >/var/home/coverage/pythonroot/pycoco-0.1-py2.5.egg/EGG-INFO/PKG-INFO >/var/home/coverage/pythonroot/pycoco-0.1-py2.5.egg/EGG-INFO/SOURCES.txt >/var/home/coverage/pythonroot/pycoco-0.1-py2.5.egg/EGG-INFO/dependency_links.txt >/var/home/coverage/pythonroot/pycoco-0.1-py2.5.egg/EGG-INFO/top_level.txt >/var/home/coverage/pythonroot/pycoco-0.1-py2.5.egg/EGG-INFO/not-zip-safe > >with no coverage.css in sight. I'm using a ~/.pydistutils.cfg, which >contains: > >[install] >install_lib = ~/pythonroot > >I've also tried including the include_package_data=True argument, moved >the CSS file out of the src directory, changes the include statement in >MANIFEST.in to "include coverage.css" and various combination thereof >always with the same result. I'm using setuptools 0.6c3. > >Am I doing something stupid? > >BTW, I've moved most of our Python packages to setuptools, and so far it >really simplifies packages handling. The only remaining problem is >setuptools+pyexe. So thanks for all your work on setuptools. > >Servus, > Walter From pje at telecommunity.com Mon Oct 2 02:44:57 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 01 Oct 2006 20:44:57 -0400 Subject: [Distutils] Limitations of setup_requires In-Reply-To: <940D723B-64E0-4213-9C23-EBBACD1F1D7A@lbl.gov> Message-ID: <5.1.1.6.0.20061001204208.029fb1e0@sparrow.telecommunity.com> At 04:47 PM 9/29/2006 -0700, Joshua Boverhof wrote: >I included the "ZSI" package in "setup_requires" and "install_requires", >but it isn't retrieved/installed before I use it via >"make_generated_packages". > >Here is what I'm doing: > >class PGDistribution(_Distribution): > """Indirection so that setuptools grabs "setup_requires" dependencies > before calling "_make_generated_package()" > """ > def finalize_options (self): > if _make_generated_package() is True: > self.packages = find_packages() > _Distribution.finalize_options(self) I don't understand what the purpose of this code is. find_packages() is not going to give you a different result by being called here, than if you just called it in setup(). From pje at telecommunity.com Mon Oct 2 02:49:41 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 01 Oct 2006 20:49:41 -0400 Subject: [Distutils] Limitations install_requires? In-Reply-To: Message-ID: <5.1.1.6.0.20061001204534.02a3c968@sparrow.telecommunity.com> At 04:50 PM 9/29/2006 -0700, Joshua Boverhof wrote: >I've had problems getting it to install these packages (python2.5, >setuptools 0.6c3): > >packages with extensions fail: > 4Suite -- > Twisted -- already declared problem in the PackageNotes > >Second, if the file name of a distribution doesn't match the >"package" name once installed setuptools fails to recognize it as >installed. > > ZopeInterface --> zope.interface > -- appears to build and install, but fails when setuptools tries to >import it as "ZopeInterface" > >Or if the package installs as a sub-package of an existing package: > > TwistedWeb --> twisted.web > -- again fails when it tries to import "TwistedWeb" > >I think the first two are currently impossible, but potentially I >missed something with the second two? Setuptools doesn't "import" ZopeInterface or TwistedWeb - it looks for *installed distributions* with those names. There's a big difference. If you find an installation package for "zope.interface" whose filename is "ZopeInterface" (and similar for TwistedWeb/twisted.web), then that is an indication that the installation package was not created using the distutils (or that it was renamed after creation). In either case, it is simply not compatible with setuptools. Setuptools only supports installing *unmodified* and *un-renamed* distutils-based distributions, although it tries very hard to cope with customized setup scripts that extend the distutils. There are limits, however, and Twisted falls well outside the scope of what setuptools can reasonably monkeypatch. :( From JRBoverhof at lbl.gov Mon Oct 2 03:20:02 2006 From: JRBoverhof at lbl.gov (Joshua Boverhof) Date: Sun, 1 Oct 2006 18:20:02 -0700 Subject: [Distutils] Limitations of setup_requires In-Reply-To: <5.1.1.6.0.20061001204208.029fb1e0@sparrow.telecommunity.com> References: <5.1.1.6.0.20061001204208.029fb1e0@sparrow.telecommunity.com> Message-ID: <1EB82806-A3D0-40F6-98F7-8C83E64489D6@lbl.gov> Yes it is since I'm "generating" code when I do "make_generated_packages". On Oct 1, 2006, at 5:44 PM, Phillip J. Eby wrote: > At 04:47 PM 9/29/2006 -0700, Joshua Boverhof wrote: >> I included the "ZSI" package in "setup_requires" and >> "install_requires", but it isn't retrieved/installed before I use >> it via "make_generated_packages". >> >> Here is what I'm doing: >> >> class PGDistribution(_Distribution): >> """Indirection so that setuptools grabs "setup_requires" >> dependencies >> before calling "_make_generated_package()" >> """ >> def finalize_options (self): >> if _make_generated_package() is True: >> self.packages = find_packages() >> _Distribution.finalize_options(self) > > I don't understand what the purpose of this code is. find_packages > () is not going to give you a different result by being called > here, than if you just called it in setup(). -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2526 bytes Desc: not available Url : http://mail.python.org/pipermail/distutils-sig/attachments/20061001/276bf3c5/attachment.bin From pje at telecommunity.com Mon Oct 2 03:40:28 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 01 Oct 2006 21:40:28 -0400 Subject: [Distutils] Limitations of setup_requires In-Reply-To: <1EB82806-A3D0-40F6-98F7-8C83E64489D6@lbl.gov> References: <5.1.1.6.0.20061001204208.029fb1e0@sparrow.telecommunity.com> <5.1.1.6.0.20061001204208.029fb1e0@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20061001213708.02760740@sparrow.telecommunity.com> At 06:20 PM 10/1/2006 -0700, Joshua Boverhof wrote: >Yes it is since I'm "generating" code when I do >"make_generated_packages". Okay, then before you call setup() in your setup script, create an instance of setuptools.dist.Distribution() that has setup_requires set, but doesn't include any packages; and don't do anything besides create it. Then, run your package generation, and finally run setup(). You do not need a custom Distribution subclass, as far as I can tell. The only disadvantage to doing it this way is that if somebody does "setup.py --help", your package is still going to do dependency downloads and code generation. But then, that also happens the way you're doing it now. From JRBoverhof at lbl.gov Mon Oct 2 03:45:18 2006 From: JRBoverhof at lbl.gov (Joshua Boverhof) Date: Sun, 1 Oct 2006 18:45:18 -0700 Subject: [Distutils] Limitations install_requires? In-Reply-To: <5.1.1.6.0.20061001204534.02a3c968@sparrow.telecommunity.com> References: <5.1.1.6.0.20061001204534.02a3c968@sparrow.telecommunity.com> Message-ID: <1EB97E35-1435-4CE1-906B-646FA8EAC118@lbl.gov> On Oct 1, 2006, at 5:49 PM, Phillip J. Eby wrote: > At 04:50 PM 9/29/2006 -0700, Joshua Boverhof wrote: >> I've had problems getting it to install these packages (python2.5, >> setuptools 0.6c3): >> >> packages with extensions fail: >> 4Suite -- >> Twisted -- already declared problem in the PackageNotes >> >> Second, if the file name of a distribution doesn't match the >> "package" name once installed setuptools fails to recognize it as >> installed. >> >> ZopeInterface --> zope.interface >> -- appears to build and install, but fails when setuptools >> tries to >> import it as "ZopeInterface" >> >> Or if the package installs as a sub-package of an existing package: >> >> TwistedWeb --> twisted.web >> -- again fails when it tries to import "TwistedWeb" >> >> I think the first two are currently impossible, but potentially I >> missed something with the second two? > > Setuptools doesn't "import" ZopeInterface or TwistedWeb - it looks for > *installed distributions* with those names. There's a big > difference. If > you find an installation package for "zope.interface" whose > filename is > "ZopeInterface" (and similar for TwistedWeb/twisted.web), then that > is an > indication that the installation package was not created using the > distutils (or that it was renamed after creation). > I'm not an expert on setuptools, I'm just guessing what it does based on behavior. Maybe this is explained somewhere in the web pages, sorry I often neglect to thoroughly read documentation. However, I did read that setuptools only supports distutils-based distributions, I was trying to ask if there were ways around this. I'll take this as a unqualified "no". > In either case, it is simply not compatible with setuptools. > Setuptools > only supports installing *unmodified* and *un-renamed* distutils-based > distributions, although it tries very hard to cope with customized > setup > scripts that extend the distutils. There are limits, however, and > Twisted > falls well outside the scope of what setuptools can reasonably > monkeypatch. :( > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig In any case, 4/5 of my dependencies are for one reason or another not compatible with setuptools. I'll ask them to change their ways, in any I'm sure this battle will be easier to fight once setuptools is part of the standard python distribution. I thought was slated for python2.5, but I've checked the release notes and I don't see mention of its inclusion. -josh -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2526 bytes Desc: not available Url : http://mail.python.org/pipermail/distutils-sig/attachments/20061001/beb748b1/attachment.bin From jeremy.kloth at 4suite.org Mon Oct 2 03:55:01 2006 From: jeremy.kloth at 4suite.org (Jeremy Kloth) Date: Sun, 1 Oct 2006 19:55:01 -0600 Subject: [Distutils] Limitations install_requires? In-Reply-To: <1EB97E35-1435-4CE1-906B-646FA8EAC118@lbl.gov> References: <5.1.1.6.0.20061001204534.02a3c968@sparrow.telecommunity.com> <1EB97E35-1435-4CE1-906B-646FA8EAC118@lbl.gov> Message-ID: <200610011955.01906.jeremy.kloth@4suite.org> On Sunday, October 1, 2006 7:45 pm, Joshua Boverhof wrote: > On Oct 1, 2006, at 5:49 PM, Phillip J. Eby wrote: > > At 04:50 PM 9/29/2006 -0700, Joshua Boverhof wrote: > >> I've had problems getting it to install these packages (python2.5, > >> setuptools 0.6c3): > >> > >> packages with extensions fail: > >> 4Suite -- > > In any case, 4/5 of my dependencies are for one reason or another not > compatible with setuptools. I'll ask them to change their ways, in > any I'm sure this battle will be easier to fight once setuptools is > part of the standard python distribution. I thought was slated for > python2.5, but I've checked the release notes and I don't see mention > of its inclusion. 4Suite goes through great lengths to support setuptools. What are the problems that you are having with it in regards to setuptools? Python 2.5 and setuptools works just fine with Linux, Mac OSX and Windows. We are getting *VERY* close to a final release and we would like to fix any problems you are having . -- Jeremy Kloth http://4suite.org/ From pje at telecommunity.com Mon Oct 2 04:05:14 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 01 Oct 2006 22:05:14 -0400 Subject: [Distutils] Limitations install_requires? In-Reply-To: <1EB97E35-1435-4CE1-906B-646FA8EAC118@lbl.gov> References: <5.1.1.6.0.20061001204534.02a3c968@sparrow.telecommunity.com> <5.1.1.6.0.20061001204534.02a3c968@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20061001215052.02b2c548@sparrow.telecommunity.com> At 06:45 PM 10/1/2006 -0700, Joshua Boverhof wrote: >In any case, 4/5 of my dependencies are for one reason or another not >compatible with setuptools. I'll ask them to change their ways, in >any I'm sure this battle will be easier to fight once setuptools is >part of the standard python distribution. I thought was slated for >python2.5, but I've checked the release notes and I don't see mention >of its inclusion. One thing was added to Python 2.5: if you install a package under 2.5 using the distutils -- assuming you haven't done some serious surgery on the distutils, that is -- then the package will be installed with an .egg-info file containing the data that setuptools needs to *detect* the package has been installed. So, as long as your users have the dependencies installed, setuptools will at least not try to install them a second time. By the way, one way around the distribution issue: if you can find a way to build .egg files for your dependencies, then you can offer them for download from your own site, and use dependency_links to point to them. EasyInstall will prefer to use pre-built eggs instead of source, which will then work around your problem. You still, of course, need to be able to build the eggs, but in the worst-case scenario, you could do this by zipping the appropriate parts of site-packages, along with some extra steps. If the package uses the distutils' standard "install" commands, it should be possible to use a trick like: setup.py install --root=/some/dir and then zip up selected subdirectories of /some/dir, adding an EGG-INFO subdirectory as well. The internal format of eggs is documented here: http://svn.python.org/view/sandbox/trunk/setuptools/doc/formats.txt?view=auto From JRBoverhof at lbl.gov Tue Oct 3 01:54:19 2006 From: JRBoverhof at lbl.gov (Joshua Boverhof) Date: Mon, 2 Oct 2006 16:54:19 -0700 Subject: [Distutils] Limitations install_requires? In-Reply-To: <200610011955.01906.jeremy.kloth@4suite.org> References: <5.1.1.6.0.20061001204534.02a3c968@sparrow.telecommunity.com> <1EB97E35-1435-4CE1-906B-646FA8EAC118@lbl.gov> <200610011955.01906.jeremy.kloth@4suite.org> Message-ID: <8708694C-9349-4AC3-B08F-D5B08988AD3E@lbl.gov> There are a couple known problems with 4Suite-1.0rc[3,4] on MacOSX, so the 4Suite trouble I was having wasn't a setuptools problem. I just tested this out on a Linux box and it works fine. -josh On Oct 1, 2006, at 6:55 PM, Jeremy Kloth wrote: > On Sunday, October 1, 2006 7:45 pm, Joshua Boverhof wrote: >> On Oct 1, 2006, at 5:49 PM, Phillip J. Eby wrote: >>> At 04:50 PM 9/29/2006 -0700, Joshua Boverhof wrote: >>>> I've had problems getting it to install these packages (python2.5, >>>> setuptools 0.6c3): >>>> >>>> packages with extensions fail: >>>> 4Suite -- >> >> In any case, 4/5 of my dependencies are for one reason or another not >> compatible with setuptools. I'll ask them to change their ways, in >> any I'm sure this battle will be easier to fight once setuptools is >> part of the standard python distribution. I thought was slated for >> python2.5, but I've checked the release notes and I don't see mention >> of its inclusion. > > 4Suite goes through great lengths to support setuptools. What are the > problems that you are having with it in regards to setuptools? > Python 2.5 > and setuptools works just fine with Linux, Mac OSX and Windows. We > are > getting *VERY* close to a final release and we would like to fix > any problems > you are having . > > -- > Jeremy Kloth > http://4suite.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2526 bytes Desc: not available Url : http://mail.python.org/pipermail/distutils-sig/attachments/20061002/4e20f442/attachment.bin From JRBoverhof at lbl.gov Tue Oct 3 03:04:56 2006 From: JRBoverhof at lbl.gov (Joshua Boverhof) Date: Mon, 2 Oct 2006 18:04:56 -0700 Subject: [Distutils] Limitations of setup_requires In-Reply-To: <5.1.1.6.0.20061001213708.02760740@sparrow.telecommunity.com> References: <5.1.1.6.0.20061001204208.029fb1e0@sparrow.telecommunity.com> <5.1.1.6.0.20061001204208.029fb1e0@sparrow.telecommunity.com> <5.1.1.6.0.20061001213708.02760740@sparrow.telecommunity.com> Message-ID: <0851E2C8-E495-4382-86D3-A971B6D77BF6@lbl.gov> I have a package specified in both "setup_requires" and "install_requires", the package is downloaded and used during setup but it is never installed in site-packages like the other entry. This is happening on both my Mac and a Linux box. This fails for me with setuptools-0.6c3, python2.5, but works with w/ python2.4 Here is what I'm doing: install_requires=[ "ZSI >= 1.2.0rc2", "4Suite-XML >= 1.0rc4", ], setup_requires=[ "ZSI >= 1.2.0rc2", ], ssh1:%python setup.py install Now if I remove it from "setup_requires" and remove the egg (otherwise it does nothing): %rm ZSI-2.0_rc2-py2.5.egg install_requires=[ "ZSI >= 1.2.0rc2", "4Suite-XML >= 1.0rc4", ], setup_requires=[ ], boverhof at pabst(341)-ssh1:%rm ZSI-2.0_rc2-py2.5.egg boverhof at pabst(342)-ssh1:%rm -rf pyGridWare.egg-info/boverhof at pabst (343)-ssh1:%python setup.py install --force It now installs. Reading http://www.python.org/pypi/ZSI/2.0-rc2 Best match: ZSI 2.0-rc2 Downloading http://cheeseshop.python.org/packages/source/Z/ZSI/ ZSI-2.0-rc2.tar.gz#md5=25b3ab7a3ff8285d51c35c223359c060 Processing ZSI-2.0-rc2.tar.gz Running ZSI-2.0-rc2/setup.py -q bdist_egg --dist-dir /tmp/ easy_install-kps9vS/ZSI-2.0-rc2/egg-dist-tmp-pQbxQp zip_safe flag not set; analyzing archive contents... Adding ZSI 2.0-rc2 to easy-install.pth file Has anyone else noticed this behavior? -josh On Oct 1, 2006, at 6:40 PM, Phillip J. Eby wrote: > At 06:20 PM 10/1/2006 -0700, Joshua Boverhof wrote: >> Yes it is since I'm "generating" code when I do >> "make_generated_packages". > > Okay, then before you call setup() in your setup script, create an > instance of setuptools.dist.Distribution() that has setup_requires > set, but doesn't include any packages; and don't do anything > besides create it. Then, run your package generation, and finally > run setup(). You do not need a custom Distribution subclass, as > far as I can tell. > > The only disadvantage to doing it this way is that if somebody does > "setup.py --help", your package is still going to do dependency > downloads and code generation. But then, that also happens the way > you're doing it now. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2526 bytes Desc: not available Url : http://mail.python.org/pipermail/distutils-sig/attachments/20061002/f3ddcd3c/attachment.bin From pje at telecommunity.com Tue Oct 3 23:31:03 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 03 Oct 2006 17:31:03 -0400 Subject: [Distutils] Draft of new setuptools installation instructions Message-ID: <5.1.1.6.0.20061003170715.026db660@sparrow.telecommunity.com> I'm working on new installation instructions for setuptools 0.6c4, which will no longer use ez_setup.py for end-user manual installation of setuptools (as opposed to bundled installation with another package). The basic idea is that the PyPI page for setuptools would contain these instructions, and links to the various setuptools documentation pages. ez_setup will still be available for as an importable module for people to integrate with their setuptools-using packages, but when run as a script, ez_setup.py will simply tell people to consult the PyPI page. I'm looking for any feedback people might have, on the installation process itself or on the directions. Some particular areas of question/concern: * Should ez_setup perhaps launch a web browser pointing to these instructions, instead of trying to download setuptools itself? (This would bypass most of those annoying proxy problems right off the bat, at least when just installing setuptools.) * Should we make more effort to create a usable command-line experience on Windows? Perhaps use a post-install script to register easy_install so it works from "Start/Run", with an automatic pause to prevent the opened console window from closing when it finishes running? Should we include optional registration for the ``.egg`` extension so downloading or double-clicking an egg installs it? * The new install procedure bypasses firewall issues for installing setuptools itself, but doesn't do anything about the issue for packages that have dependencies. Is there anything else that can be done about that? Anyway, those are the questions; here are the docs so far: ------------------------- Installation Instructions ------------------------- Windows ======= Install setuptools using the provided ``.exe`` installer. If you've previously installed older versions of setuptools, please delete all ``setuptools*.egg`` files from your system FIRST. If you are upgrading a previous version of setuptools that was installed using an ``.exe`` installer, please be sure to also uninstall that older version via your system's "Add/Remove Programs" feature, BEFORE installing the newer version. RPM-Based Systems ================= Install setuptools using the provided source RPM. The included ``.spec`` file assumes you are installing using the default ``python`` executable, and is thus not specific to a particular Python version. The ``easy_install`` executable will be installed to a system ``bin`` directory such as ``/usr/bin``. If you wish to install to a custom installation location, please use the ``.egg`` installation approach described in the following section. Cygwin, Mac OS X, Linux, Other ============================== 1. Download the appropriate egg for your version of Python (e.g. ``setuptools-0.6c4-py2.4.egg``). Do NOT rename it. 2. Make it executable (e.g. ``chmod +x setuptools-0.6c4-py2.4.egg``) 3. Run it (e.g. ``./setuptools-0.6c4-py2.4.egg``). Setuptools will install itself using the matching version of Python (e.g. ``python2.4``), and will place the ``easy_install`` executable in the default location for installing Python scripts. If you want to install setuptools to somewhere other than ``site-packages`` or your default distutils installation locations for libraries and scripts, you may use EasyInstall command-line options such as ``--prefix``, ``--install-dir``, and so on. You can use ``--help`` to get a full options list, but we recommend consulting the `EasyInstall manual`_ for detailed instructions, especially `the section on custom installation locations`_. Cygwin Note ----------- If you are trying to install setuptools for the *Windows* version of Python (as opposed to the Cygwin version that lives in ``/usr/bin``), you must make sure that an appropriate executable (``python2.3``, ``python2.4``, or ``python2.5``) is on your ``PATH`` when invoking the egg. For example, doing the following at a Cygwin bash prompt will install setuptools for the *Windows* Python found at ``C:\\Python24``:: ln -s /cygdrive/c/Python24/python.exe python2.4 PATH=.:$PATH ./setuptools-0.6c4-py2.4.egg rm python2.4 From pje at telecommunity.com Tue Oct 3 23:37:50 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 03 Oct 2006 17:37:50 -0400 Subject: [Distutils] Limitations of setup_requires In-Reply-To: <0851E2C8-E495-4382-86D3-A971B6D77BF6@lbl.gov> References: <5.1.1.6.0.20061001213708.02760740@sparrow.telecommunity.com> <5.1.1.6.0.20061001204208.029fb1e0@sparrow.telecommunity.com> <5.1.1.6.0.20061001204208.029fb1e0@sparrow.telecommunity.com> <5.1.1.6.0.20061001213708.02760740@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20061003173549.026db318@sparrow.telecommunity.com> At 06:04 PM 10/2/2006 -0700, Joshua Boverhof wrote: >I have a package specified in both "setup_requires" and >"install_requires", the package is downloaded and used during setup >but it is never installed in site-packages like the other entry. >This is happening on both my Mac and a Linux box. > >This fails for me with setuptools-0.6c3, python2.5, but works with w/ >python2.4 Interesting. My one theory of how this problem might be happening, should cause it to fail on *all* versions of Python. Are you sure that you don't just happen to already have ZSI installed in your python 2.4 site-packages directory? Thanks. From bob at redivi.com Tue Oct 3 23:40:13 2006 From: bob at redivi.com (Bob Ippolito) Date: Tue, 3 Oct 2006 14:40:13 -0700 Subject: [Distutils] Draft of new setuptools installation instructions In-Reply-To: <5.1.1.6.0.20061003170715.026db660@sparrow.telecommunity.com> References: <5.1.1.6.0.20061003170715.026db660@sparrow.telecommunity.com> Message-ID: <6a36e7290610031440l5e349be4o1e9b8a454eb0150a@mail.gmail.com> On 10/3/06, Phillip J. Eby wrote: > > Cygwin, Mac OS X, Linux, Other > ============================== > > 1. Download the appropriate egg for your version of Python (e.g. > ``setuptools-0.6c4-py2.4.egg``). Do NOT rename it. > > 2. Make it executable (e.g. ``chmod +x setuptools-0.6c4-py2.4.egg``) > > 3. Run it (e.g. ``./setuptools-0.6c4-py2.4.egg``). Setuptools will > install itself using the matching version of Python (e.g. > ``python2.4``), and will place the ``easy_install`` executable > in the default location for installing Python scripts. > > If you want to install setuptools to somewhere other than ``site-packages`` > or your default distutils installation locations for libraries and scripts, you > may use EasyInstall command-line options such as ``--prefix``, > ``--install-dir``, and so on. You can use ``--help`` to get a full options > list, but we recommend consulting the `EasyInstall manual`_ for detailed > instructions, especially `the section on custom installation locations`_. Wouldn't it be easier to just tell them to do "sh ./setuptools-0.6c4-py2.4.egg" instead of marking it executable first? -bob From p.f.moore at gmail.com Tue Oct 3 23:43:29 2006 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 3 Oct 2006 22:43:29 +0100 Subject: [Distutils] Draft of new setuptools installation instructions In-Reply-To: <5.1.1.6.0.20061003170715.026db660@sparrow.telecommunity.com> References: <5.1.1.6.0.20061003170715.026db660@sparrow.telecommunity.com> Message-ID: <79990c6b0610031443q399822ffw182042ba3c164dc0@mail.gmail.com> On 10/3/06, Phillip J. Eby wrote: > I'm looking for any feedback people might have, on the installation process > itself or on the directions. Some particular areas of question/concern: Looks fine for Windows. > * Should we make more effort to create a usable command-line experience on > Windows? Perhaps use a post-install script to register easy_install so it > works from "Start/Run", with an automatic pause to prevent the opened > console window from closing when it finishes running? Should we include > optional registration for the ``.egg`` extension so downloading or > double-clicking an egg installs it? IMHO, no. Your particular suggestions would annoy me immensely, but I'm sure others would like them (and would hate whatever I'd prefer). There is no solution that will please everyone (or even a majority, in all probability). Better to do what python itself does, and *not* try to integrate too closely, leaving it to command line users to customize their own environment. > * The new install procedure bypasses firewall issues for installing > setuptools itself, but doesn't do anything about the issue for packages > that have dependencies. Is there anything else that can be done about that? I've not worked with enough such packages to know the answer to this without asking, but is there an incantation which says "check dependencies, if all are present install, otherwise list the missing dependencies and stop". If that's available, users can manually download what they need. That is minimal, but effective - in my view, simplicity is a virtue here. Paul. From pje at telecommunity.com Tue Oct 3 23:46:18 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 03 Oct 2006 17:46:18 -0400 Subject: [Distutils] Draft of new setuptools installation instructions In-Reply-To: <6a36e7290610031440l5e349be4o1e9b8a454eb0150a@mail.gmail.co m> References: <5.1.1.6.0.20061003170715.026db660@sparrow.telecommunity.com> <5.1.1.6.0.20061003170715.026db660@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20061003174335.02867df8@sparrow.telecommunity.com> At 02:40 PM 10/3/2006 -0700, Bob Ippolito wrote: >Wouldn't it be easier to just tell them to do "sh >./setuptools-0.6c4-py2.4.egg" instead of marking it executable first? I found the phrasing much more awkward that way, as it led to having to explain the idea that it's got a shell script in there. It seemed easier to just dodge the issue. But if you can come up with a phrasing that works, go for it. From bob at redivi.com Tue Oct 3 23:55:14 2006 From: bob at redivi.com (Bob Ippolito) Date: Tue, 3 Oct 2006 14:55:14 -0700 Subject: [Distutils] Draft of new setuptools installation instructions In-Reply-To: <5.1.1.6.0.20061003174335.02867df8@sparrow.telecommunity.com> References: <5.1.1.6.0.20061003170715.026db660@sparrow.telecommunity.com> <5.1.1.6.0.20061003174335.02867df8@sparrow.telecommunity.com> Message-ID: <6a36e7290610031455v3338309ane368e1bf2ddc4703@mail.gmail.com> On 10/3/06, Phillip J. Eby wrote: > At 02:40 PM 10/3/2006 -0700, Bob Ippolito wrote: > >Wouldn't it be easier to just tell them to do "sh > >./setuptools-0.6c4-py2.4.egg" instead of marking it executable first? > > I found the phrasing much more awkward that way, as it led to having to > explain the idea that it's got a shell script in there. It seemed easier > to just dodge the issue. But if you can come up with a phrasing that > works, go for it. > Does it really require explanation? 2) Run it as a shell script (e.g. ``sh ./setuptools-0.6c4-py2.4.egg``). Setuptools will install itself using the matching version of Python (e.g. ``python2.4``), and will place the ``easy_install`` executable in the default location for installing Python scripts. -bob From pje at telecommunity.com Tue Oct 3 23:56:27 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 03 Oct 2006 17:56:27 -0400 Subject: [Distutils] Draft of new setuptools installation instructions In-Reply-To: <79990c6b0610031443q399822ffw182042ba3c164dc0@mail.gmail.co m> References: <5.1.1.6.0.20061003170715.026db660@sparrow.telecommunity.com> <5.1.1.6.0.20061003170715.026db660@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20061003174745.026c4ac8@sparrow.telecommunity.com> At 10:43 PM 10/3/2006 +0100, Paul Moore wrote: >On 10/3/06, Phillip J. Eby wrote: >>Should we make more effort to create a usable command-line experience on >>Windows? Perhaps use a post-install script to register easy_install so it >>works from "Start/Run", with an automatic pause to prevent the opened >>console window from closing when it finishes running? Should we include >>optional registration for the ``.egg`` extension so downloading or >>double-clicking an egg installs it? > >IMHO, no. Your particular suggestions would annoy me immensely, Are you sure? Two points here, that I think you're not noticing: 1. Registering an "App Paths" entry for easy_install would simply make it *possible* to type "easy_install" in the Start/Run box and have it work. It wouldn't force you to use it, and seems quite unlikely to conflict with anything else. (It could also be made optional, as per point #2.) 2. I said *optional* registration of the ``.egg`` extension; the idea would be that you'd have a dialog come up at the end of the .win32.exe run, to ask you if you wanted to add it. So, unless you're saying that having the *options* to do these things would annoy you, I don't understand your statement. >>* The new install procedure bypasses firewall issues for installing >>setuptools itself, but doesn't do anything about the issue for packages >>that have dependencies. Is there anything else that can be done about that? > >I've not worked with enough such packages to know the answer to this >without asking, but is there an incantation which says "check >dependencies, if all are present install, otherwise list the missing >dependencies and stop". If that's available, users can manually >download what they need. That is minimal, but effective - in my view, >simplicity is a virtue here. Well, keep in mind that such a thing would only work for *one* level of dependencies; if your dependencies have dependencies you'll go through that repeatedly. But I could perhaps add a --no-deps option to "install" that would do the trick. easy_install already has a --no-deps option, but it currently doesn't *display* the dependency list; it probably should. It also doesn't stop the installation process. One minor issue: dependency checking currently takes place *after* the primary egg is installed, and changing that is too big a refactoring to happen in the remaining time until the 0.6 release. But it's definitely something to think about for 0.7. From p.f.moore at gmail.com Wed Oct 4 00:12:40 2006 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 3 Oct 2006 23:12:40 +0100 Subject: [Distutils] Draft of new setuptools installation instructions In-Reply-To: <5.1.1.6.0.20061003174745.026c4ac8@sparrow.telecommunity.com> References: <5.1.1.6.0.20061003170715.026db660@sparrow.telecommunity.com> <5.1.1.6.0.20061003174745.026c4ac8@sparrow.telecommunity.com> Message-ID: <79990c6b0610031512y5c4fc831l39269263239a437e@mail.gmail.com> On 10/3/06, Phillip J. Eby wrote: > >IMHO, no. Your particular suggestions would annoy me immensely, > > Are you sure? Two points here, that I think you're not noticing: > > 1. Registering an "App Paths" entry for easy_install would simply make it > *possible* to type "easy_install" in the Start/Run box and have it > work. It wouldn't force you to use it, and seems quite unlikely to > conflict with anything else. (It could also be made optional, as per point > #2.) I had not understood that you meant to use "App Paths". That's reasonable (it's what Python does, for example). I also misread your statement as implying you would add a shortcut. Also, the automatic pause would annoy me. How would you make it *not* happen if I ran the command from a console window I already had open? > 2. I said *optional* registration of the ``.egg`` extension; the idea would > be that you'd have a dialog come up at the end of the .win32.exe run, to > ask you if you wanted to add it. I noticed that, but assumed you'd want to make the default "yes". I dislike options that I *always* want to switch off. But I retract the "immensely" bit. Overreaction there, sorry :-) > Well, keep in mind that such a thing would only work for *one* level of > dependencies; if your dependencies have dependencies you'll go through that > repeatedly. Clearly. That seems implicit (and hence, not a problem) to me in the fact that you aren't getting the dependencies, so how could you scan them for further dependencies. Again, it fits my "keep it simple" attitude. Paul. From bear42 at code-bear.com Wed Oct 4 00:14:27 2006 From: bear42 at code-bear.com (bear) Date: Tue, 03 Oct 2006 18:14:27 -0400 Subject: [Distutils] Draft of new setuptools installation instructions In-Reply-To: <5.1.1.6.0.20061003170715.026db660@sparrow.telecommunity.com> References: <5.1.1.6.0.20061003170715.026db660@sparrow.telecommunity.com> Message-ID: <4522E0C3.4080300@code-bear.com> Phillip J. Eby wrote: > Windows > ======= > > Install setuptools using the provided ``.exe`` installer. If you've > previously installed older versions of setuptools, please delete all > ``setuptools*.egg`` files from your system FIRST. > > If you are upgrading a previous version of setuptools that was installed > using an ``.exe`` installer, please be sure to also uninstall that older > version via your system's "Add/Remove Programs" feature, BEFORE installing > the newer version. > This works for me. The extra check boxes at the end (or beginning) of the install to set .egg extension information also work for me as long as the default is "no" > RPM-Based Systems > ================= > > Install setuptools using the provided source RPM. The included ``.spec`` > file assumes you are installing using the default ``python`` executable, > and is thus not specific to a particular Python version. The > ``easy_install`` executable will be installed to a system ``bin`` directory > such as ``/usr/bin``. > > If you wish to install to a custom installation location, please use the > ``.egg`` installation approach described in the following section. > you should be able to create a spec file that allows for a --prefix option but that's a nit > Cygwin, Mac OS X, Linux, Other > ============================== > > 1. Download the appropriate egg for your version of Python (e.g. > ``setuptools-0.6c4-py2.4.egg``). Do NOT rename it. > > 2. Make it executable (e.g. ``chmod +x setuptools-0.6c4-py2.4.egg``) > I have to agree with the earlier comments that having a chmod step seems odd - just saying "run the egg as a shell script (e.g. sh setuptools)..." is clear and concise --- bear From pje at telecommunity.com Wed Oct 4 01:08:03 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 03 Oct 2006 19:08:03 -0400 Subject: [Distutils] Draft of new setuptools installation instructions In-Reply-To: <79990c6b0610031512y5c4fc831l39269263239a437e@mail.gmail.co m> References: <5.1.1.6.0.20061003174745.026c4ac8@sparrow.telecommunity.com> <5.1.1.6.0.20061003170715.026db660@sparrow.telecommunity.com> <5.1.1.6.0.20061003174745.026c4ac8@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20061003190616.026ec0d0@sparrow.telecommunity.com> At 11:12 PM 10/3/2006 +0100, Paul Moore wrote: >Also, the automatic pause would annoy me. How would you make it *not* >happen if I ran the command from a console window I already had open? The idea would be to register a separate "easy_install-win.exe" under "easy_install" in App Paths. This version would pause after termination, while the normal version would not. Since console programs invoked via the App Paths mechanism *always* get their own new window, this should work out nicely. From pje at telecommunity.com Wed Oct 4 01:11:14 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 03 Oct 2006 19:11:14 -0400 Subject: [Distutils] Draft of new setuptools installation instructions In-Reply-To: <4522E0C3.4080300@code-bear.com> References: <5.1.1.6.0.20061003170715.026db660@sparrow.telecommunity.com> <5.1.1.6.0.20061003170715.026db660@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20061003190929.0281cc88@sparrow.telecommunity.com> At 06:14 PM 10/3/2006 -0400, bear wrote: >Phillip J. Eby wrote: >>RPM-Based Systems >>================= >> >>Install setuptools using the provided source RPM. The included ``.spec`` >>file assumes you are installing using the default ``python`` executable, >>and is thus not specific to a particular Python version. The >>``easy_install`` executable will be installed to a system ``bin`` >>directory such as ``/usr/bin``. >> >>If you wish to install to a custom installation location, please use the >>``.egg`` installation approach described in the following section. >> >you should be able to create a spec file that allows for a --prefix option >but that's a nit Actually, I meant installation locations other than ``site-packages``, not ``site-packages`` with a different prefix. I'll edit this to clarify that better. From pje at telecommunity.com Wed Oct 4 02:10:49 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 03 Oct 2006 20:10:49 -0400 Subject: [Distutils] More Windows questions Message-ID: <5.1.1.6.0.20061003193456.02d20eb0@sparrow.telecommunity.com> Okay, I've looked at the registry a bit, and it seems like I can register App Paths either under HKLM (for the whole machine) or under HKCU (for the current user). However, both seem to have issues. Under later versions of Windows, especially when not running with administrative privileges, the HKLM keys will probably be off limits, and might generate warning popups of various kinds. On the other hand, HKCU might not be that useful for a system-wide Python installation, and would seem to force you to repeatedly reinstall for each user. I could ask the user, but they're not really any more likely to know what to do either, and in any case I'm thinking that for 0.6 I'd prefer to keep the installation code as simple as possible. (Which means I don't want to even bother with a dialog, and will therefore just skip the .egg registration idea.) Examining this a little further, I notice that there are a couple of problems with setuptools generating bdist_wininst installers right now: 1. The "-script.py" files don't get their #! lines updated, so if the destination machine has Python in a different location than on the generating machine, the wrapper .exe's won't work. 2. If you generate a bdist_wininst on a non-Windows platform, you don't get the "-script.py" and ".exe" wrappers to begin with. So, here's what I'm thinking I should do. First, setuptools' bdist_wininst should always generate -script.py and .exe wrappers, so that you can build correctly on non-Windows platforms. Second, setuptools' bdist_wininst should add its own postinstall script that: 1. Updates the #! lines in the installed scripts to point to the right executable 2. Register HKCU "App Paths" entries for all console scripts, pointing to a second '-conio-.exe' wrapper for the same '-script.py', thus allowing console scripts to be run from the "Start/Run" command line. (Note that easy_install could and maybe should do the same when run on Windows, but could have a command-line option to disable it, like --no-register or something.) The net result would be that on Windows, the command-line would become at least somewhat useful. If you are already in a command shell, "start foo args" will run the console script in a new window, without needing you to modify PATH or the default installation location(s). This isn't perfect, but as an "out of the box" usability experience for the Windows command line, it would beat the heck out of what we have now. Any thoughts? From chris at kateandchris.net Wed Oct 4 06:15:30 2006 From: chris at kateandchris.net (Chris Lambacher) Date: Wed, 4 Oct 2006 00:15:30 -0400 Subject: [Distutils] More Windows questions In-Reply-To: <5.1.1.6.0.20061003193456.02d20eb0@sparrow.telecommunity.com> References: <5.1.1.6.0.20061003193456.02d20eb0@sparrow.telecommunity.com> Message-ID: <20061004041530.GA5903@kateandchris.net> On Tue, Oct 03, 2006 at 08:10:49PM -0400, Phillip J. Eby wrote: > Okay, I've looked at the registry a bit, and it seems like I can register > App Paths either under HKLM (for the whole machine) or under HKCU (for the > current user). However, both seem to have issues. > > Under later versions of Windows, especially when not running with > administrative privileges, the HKLM keys will probably be off limits, and > might generate warning popups of various kinds. On the other hand, HKCU > might not be that useful for a system-wide Python installation, and would > seem to force you to repeatedly reinstall for each user. > > I could ask the user, but they're not really any more likely to know what > to do either, and in any case I'm thinking that for 0.6 I'd prefer to keep I have seen a number of installers that phrase it as a radio group like "Install for: [] all users [] just me.". You can probably detect whether the user has admin rights and not ask if they don't have permission for the initial install of setuptools. Eggs installed with easy_install can know how easy install was installed and follow that setting if the user doing the install is the same or has equivalent rights. I think there are 3 scenarios on Windows that are likely: 1. The installing user is always a privileged user, and the only user of the computer so either HKLM and HKCU will work 2. The installing user is unprivileged and therefore not expecting to install for everyone, so HKCU will do the trick. 3. The installing user is the administrator who expects the installation to work for all users and so needs HKLM to work. Presumably this user knows something about Windows (not necessarily so for the first two users) but is far less likely to know anything about windows (i.e. is installing for user #2 and doesn't know anything about this Python crap, he just wants user #2 to stop bugging him about installing program/library/package x). > the installation code as simple as possible. (Which means I don't want to > even bother with a dialog, and will therefore just skip the .egg > registration idea.) If you (can) detect permissions doing what you can with those permission will probably "just work". I don't know how you go about figuring out what permissions the current user has. You might need ctypes or win32all or a C extension to figure out what permissions a user has. On the download side, is there a way to force the download to go through the windows APIs that perform the proxy authentication? I think the Cygwin installer does something like that. I always used http://ntlmaps.sourceforge.net/ which is written in python and licensed under GPL. Maybe that can be used, unless I misunderstand the download issue. -Chris From pje at telecommunity.com Wed Oct 4 06:12:32 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 04 Oct 2006 00:12:32 -0400 Subject: [Distutils] More Windows questions In-Reply-To: <20061004041530.GA5903@kateandchris.net> References: <5.1.1.6.0.20061003193456.02d20eb0@sparrow.telecommunity.com> <5.1.1.6.0.20061003193456.02d20eb0@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20061004000625.026d5e48@sparrow.telecommunity.com> At 12:15 AM 10/4/2006 -0400, Chris Lambacher wrote: >You might need ctypes or win32all or a C >extension to figure out what permissions a user has. Well, I suppose I could just try modifying the registry. :) The big question is whether that causes a bunch of warnings to pop up, or triggers antivirus programs. Any thoughts? >On the download side, is there a way to force the download to go through the >windows APIs that perform the proxy authentication? I think the Cygwin >installer does something like that. I always used >http://ntlmaps.sourceforge.net/ which is written in python and licensed under >GPL. ...which effectively makes it incompatible with setuptools' licensing. > Maybe that can be used, unless I misunderstand the download issue. The issue is that you still have to tell MAPS where to find the proxy settings, as well as the authentication info to use, even if you run it. However, if someone wants to create a way to run a MAPS plugin for setuptools, I'll add entry point support so that it can be plugged in independently. That way, somebody can make a plugin egg that uses MAPS to provide proxy support, and *that* library would have to be GPL, but setuptools would not because it's just providing a generic customization hook for installed programs to be told when setuptools is about to do network access. One could possibly even write some crazy plugin that actually used IE via COM to do the same thing, come to think of it, or to provide a GUI interface for downloading. :) From chris at kateandchris.net Wed Oct 4 06:59:50 2006 From: chris at kateandchris.net (Chris Lambacher) Date: Wed, 4 Oct 2006 00:59:50 -0400 Subject: [Distutils] More Windows questions In-Reply-To: <5.1.1.6.0.20061004000625.026d5e48@sparrow.telecommunity.com> References: <5.1.1.6.0.20061003193456.02d20eb0@sparrow.telecommunity.com> <5.1.1.6.0.20061003193456.02d20eb0@sparrow.telecommunity.com> <5.1.1.6.0.20061004000625.026d5e48@sparrow.telecommunity.com> Message-ID: <20061004045950.GB5903@kateandchris.net> On Wed, Oct 04, 2006 at 12:12:32AM -0400, Phillip J. Eby wrote: > At 12:15 AM 10/4/2006 -0400, Chris Lambacher wrote: > >You might need ctypes or win32all or a C > >extension to figure out what permissions a user has. > > Well, I suppose I could just try modifying the registry. :) The big > question is whether that causes a bunch of warnings to pop up, or triggers > antivirus programs. Any thoughts? It looks like _winreg raises an exception if the process does not have sufficient privileges. On vanilla XP I expect that is all that happens. Who knows what kind of wacky things 3rd party security programs do. I would think it would be really annoying if some security program popped up a dialog every time some process tried to do something it did not have the privileges for. Sounds like Vista is going to have just that sort of annoyance though some of that is probably anti-MS FUD. > > > >On the download side, is there a way to force the download to go through > >the > >windows APIs that perform the proxy authentication? I think the Cygwin > >installer does something like that. I always used > >http://ntlmaps.sourceforge.net/ which is written in python and licensed > >under > >GPL. > > ...which effectively makes it incompatible with setuptools' licensing. > > > > Maybe that can be used, unless I misunderstand the download issue. > > The issue is that you still have to tell MAPS where to find the proxy > settings, as well as the authentication info to use, even if you run it. > > However, if someone wants to create a way to run a MAPS plugin for > setuptools, I'll add entry point support so that it can be plugged in > independently. That way, somebody can make a plugin egg that uses MAPS to > provide proxy support, and *that* library would have to be GPL, but > setuptools would not because it's just providing a generic customization > hook for installed programs to be told when setuptools is about to do > network access. > > One could possibly even write some crazy plugin that actually used IE via > COM to do the same thing, come to think of it, or to provide a GUI > interface for downloading. :) You might also want to think about a "just download, don't install" option, with the follow on of pretend I am running x version of Python on y platform, so that it would be possible to just go an get all the dependencies of a particular egg. -Chris From p.f.moore at gmail.com Wed Oct 4 10:23:28 2006 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 4 Oct 2006 09:23:28 +0100 Subject: [Distutils] Draft of new setuptools installation instructions In-Reply-To: <5.1.1.6.0.20061003190616.026ec0d0@sparrow.telecommunity.com> References: <5.1.1.6.0.20061003170715.026db660@sparrow.telecommunity.com> <5.1.1.6.0.20061003174745.026c4ac8@sparrow.telecommunity.com> <5.1.1.6.0.20061003190616.026ec0d0@sparrow.telecommunity.com> Message-ID: <79990c6b0610040123p6f360f38g2e786dbf9fd924ae@mail.gmail.com> On 10/4/06, Phillip J. Eby wrote: > At 11:12 PM 10/3/2006 +0100, Paul Moore wrote: > >Also, the automatic pause would annoy me. How would you make it *not* > >happen if I ran the command from a console window I already had open? > > The idea would be to register a separate "easy_install-win.exe" under > "easy_install" in App Paths. This version would pause after termination, > while the normal version would not. Since console programs invoked via the > App Paths mechanism *always* get their own new window, this should work out > nicely. Sadly, not true. The replacement console program 4NT (which I use pretty much exclusively) uses App Paths to locate executables. So if you (in effect) alias easy_install to easy_install-win, you'll break it for 4NT users. And in a way that will confuse the heck out of them, because it's very subtle (mapping an executable name to a different executable via App Paths is not normal usage). Why not just have the second executable with a simple to type name (easy_installw, say) and suggest people type *that* in the Run window? Keep it simple! Paul. From p.f.moore at gmail.com Wed Oct 4 10:31:51 2006 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 4 Oct 2006 09:31:51 +0100 Subject: [Distutils] More Windows questions In-Reply-To: <5.1.1.6.0.20061003193456.02d20eb0@sparrow.telecommunity.com> References: <5.1.1.6.0.20061003193456.02d20eb0@sparrow.telecommunity.com> Message-ID: <79990c6b0610040131x1c2bd88ud86fcca3d5cb88fa@mail.gmail.com> On 10/4/06, Phillip J. Eby wrote: > Okay, I've looked at the registry a bit, and it seems like I can register > App Paths either under HKLM (for the whole machine) or under HKCU (for the > current user). However, both seem to have issues. [...] > 2. Register HKCU "App Paths" entries for all console scripts, pointing to a > second '-conio-.exe' wrapper for the same '-script.py', thus allowing > console scripts to be run from the "Start/Run" command line. As I said in a previous email, keep away from App Paths, and in particular, don't register an executable under any name but its own. There's very little documentation that I could find on what App Paths should do, so all you've got to go on is experiment and looking at what others do. No-one that I have seen ever registers an exe under a different name. And some programs (4NT, for example) expect the App Path name to match the exe name (and give a broken user experience if that isn't true). Essentially, you're well into "undocumented behaviour" territory there, and things *will* go wrong. Frankly, anyone using Start/Run *knows* that command line programs don't leave their window open. They know to start cmd, and type the command into that (even if they don't do anything else in cmd). Their expectations are right, don't break them by trying to be too clever. Paul. From jim at zope.com Wed Oct 4 13:15:18 2006 From: jim at zope.com (Jim Fulton) Date: Wed, 04 Oct 2006 07:15:18 -0400 Subject: [Distutils] Draft of new setuptools installation instructions In-Reply-To: <4523917D.9030207@zope.com> References: <5.1.1.6.0.20061003170715.026db660@sparrow.telecommunity.com> <4523917D.9030207@zope.com> Message-ID: <452397C6.90506@zope.com> Jim Fulton wrote: ... > This will probably break buildout's bootstrapping script, which uses > ez_setup. I was too hasty in making this remark. The buildout bootstrap script, http://svn.zope.org/zc.buildout/trunk/bootstrap/bootstrap.py?view=markup Uses the use_setup function from ez_setup.py and, as this will still be supported, there is reason to hope that this won't be broken. Jim -- Jim Fulton mailto:jim at zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From jorge.vargas at gmail.com Wed Oct 4 18:17:31 2006 From: jorge.vargas at gmail.com (Jorge Vargas) Date: Wed, 4 Oct 2006 12:17:31 -0400 Subject: [Distutils] Draft of new setuptools installation instructions In-Reply-To: <5.1.1.6.0.20061003174745.026c4ac8@sparrow.telecommunity.com> References: <5.1.1.6.0.20061003170715.026db660@sparrow.telecommunity.com> <5.1.1.6.0.20061003174745.026c4ac8@sparrow.telecommunity.com> Message-ID: <32822fe60610040917l7d63fc27k7af2d58b778fe5f8@mail.gmail.com> On 10/3/06, Phillip J. Eby wrote: > At 10:43 PM 10/3/2006 +0100, Paul Moore wrote: > >On 10/3/06, Phillip J. Eby wrote: > >>Should we make more effort to create a usable command-line experience on > >>Windows? Perhaps use a post-install script to register easy_install so it > >>works from "Start/Run", with an automatic pause to prevent the opened > >>console window from closing when it finishes running? Should we include > >>optional registration for the ``.egg`` extension so downloading or > >>double-clicking an egg installs it? > > > >IMHO, no. Your particular suggestions would annoy me immensely, > > Are you sure? Two points here, that I think you're not noticing: > > 1. Registering an "App Paths" entry for easy_install would simply make it > *possible* to type "easy_install" in the Start/Run box and have it > work. It wouldn't force you to use it, and seems quite unlikely to > conflict with anything else. (It could also be made optional, as per point > #2.) > I'll like that, on windows normally I open cmd then run easy_install, very annoying. if I undestand it correctly will be the same that python.exe does. > 2. I said *optional* registration of the ``.egg`` extension; the idea would > be that you'd have a dialog come up at the end of the .win32.exe run, to > ask you if you wanted to add it. > > So, unless you're saying that having the *options* to do these things would > annoy you, I don't understand your statement. > +1 > > >>* The new install procedure bypasses firewall issues for installing > >>setuptools itself, but doesn't do anything about the issue for packages > >>that have dependencies. Is there anything else that can be done about that? great feature thanks. > > > >I've not worked with enough such packages to know the answer to this > >without asking, but is there an incantation which says "check > >dependencies, if all are present install, otherwise list the missing > >dependencies and stop". If that's available, users can manually > >download what they need. That is minimal, but effective - in my view, > >simplicity is a virtue here. > that sounds like Red Hat dependency hell to me. > Well, keep in mind that such a thing would only work for *one* level of > dependencies; if your dependencies have dependencies you'll go through that > repeatedly. > > But I could perhaps add a --no-deps option to "install" that would do the > trick. easy_install already has a --no-deps option, but it currently > doesn't *display* the dependency list; it probably should. It also doesn't > stop the installation process. > > One minor issue: dependency checking currently takes place *after* the > primary egg is installed, and changing that is too big a refactoring to > happen in the remaining time until the 0.6 release. But it's definitely > something to think about for 0.7. > I don't see why that's a problem you should install setuptools then get the deps of all the other packages, that's why everything depends on setuptools in a implicit way. > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > From lxander.m at gmail.com Wed Oct 4 20:08:03 2006 From: lxander.m at gmail.com (Alexander Michael) Date: Wed, 4 Oct 2006 14:08:03 -0400 Subject: [Distutils] Transitioning a Python Library on a Shared Network Drive to eggs/setuptools Message-ID: <525f23e80610041108h79c4bbeema4224866767a31a3@mail.gmail.com> In the past I've managed a shared library of Python packages by using distutils to install them in secondary Library and Scripts directories on a shared network drive. This worked fine, even in our multi-platform environment. With advent of eggs, however, the secondary Library directory must be a formal "Site Directory" and not just on sys.path. The extra delay caused by the layer of network has caused simply getting --help for a simple script to take almost three seconds when it previously only took a tenth of a second. Some scripts that use many packages installed as eggs on the network drive can take as many as 8 seconds just to display the help message. I would like to install architecture independent Python packages in a single shared location so that everyone using that location is automatically upgraded. The in-house packages are modified about five times a day on average. I would like to take advantage of setuptools versioning (thus using the pkg_resources mechanisms) so deprecated portions of the system can be kept intact in some frozen state of development without having to include the version number in the package name explicitly (i.e. mymod, mymod2, .., mymod42). What is the recommended way of using eggs in such an environment? Your thoughts and advice are appreciated, Alex -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20061004/e65555e6/attachment.htm From ianb at colorstudy.com Wed Oct 4 20:25:50 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Wed, 04 Oct 2006 13:25:50 -0500 Subject: [Distutils] setuptools: but with svn-tag-revision and uncommitted project Message-ID: <4523FCAE.8000208@colorstudy.com> I get this exception when I run "python setup.py egg_info" with --tag-svn-revision and when the project has never been committed (and hence there's no committed-rev). I think it should instead give 0, or at least a good error message. (I don't see any problem with 0) Traceback (most recent call last): File "setup.py", line 24, in ? entry_points=""" File "/usr/lib/python2.4/distutils/core.py", line 149, in setup dist.run_commands() File "/usr/lib/python2.4/distutils/dist.py", line 946, in run_commands self.run_command(cmd) File "/usr/lib/python2.4/distutils/dist.py", line 965, in run_command cmd_obj.ensure_finalized() File "/usr/lib/python2.4/distutils/cmd.py", line 117, in ensure_finalized self.finalize_options() File "/usr/local/lib/python2.4/site-packages/setuptools-0.6c3-py2.4.egg/setuptools/command/egg_info.py", line 85, in finalize_options self.vtags = self.tags() File "/usr/local/lib/python2.4/site-packages/setuptools-0.6c3-py2.4.egg/setuptools/command/egg_info.py", line 179, in tags ): version += '-r%s' % self.get_svn_revision() File "/usr/local/lib/python2.4/site-packages/setuptools-0.6c3-py2.4.egg/setuptools/command/egg_info.py", line 227, in get_svn_revision localrev = max([int(m.group(1)) for m in revre.finditer(data)]) ValueError: max() arg is an empty sequence -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From chris at kateandchris.net Wed Oct 4 21:26:33 2006 From: chris at kateandchris.net (Chris Lambacher) Date: Wed, 4 Oct 2006 15:26:33 -0400 Subject: [Distutils] Draft of new setuptools installation instructions In-Reply-To: <79990c6b0610040123p6f360f38g2e786dbf9fd924ae@mail.gmail.com> References: <5.1.1.6.0.20061003170715.026db660@sparrow.telecommunity.com> <5.1.1.6.0.20061003174745.026c4ac8@sparrow.telecommunity.com> <5.1.1.6.0.20061003190616.026ec0d0@sparrow.telecommunity.com> <79990c6b0610040123p6f360f38g2e786dbf9fd924ae@mail.gmail.com> Message-ID: <20061004192633.GA8063@kateandchris.net> On Wed, Oct 04, 2006 at 09:23:28AM +0100, Paul Moore wrote: > On 10/4/06, Phillip J. Eby wrote: > > At 11:12 PM 10/3/2006 +0100, Paul Moore wrote: > > >Also, the automatic pause would annoy me. How would you make it *not* > > >happen if I ran the command from a console window I already had open? > > > > The idea would be to register a separate "easy_install-win.exe" under > > "easy_install" in App Paths. This version would pause after termination, > > while the normal version would not. Since console programs invoked via the > > App Paths mechanism *always* get their own new window, this should work out > > nicely. > > Sadly, not true. The replacement console program 4NT (which I use > pretty much exclusively) uses App Paths to locate executables. So if > you (in effect) alias easy_install to easy_install-win, you'll break > it for 4NT users. And in a way that will confuse the heck out of them, > because it's very subtle (mapping an executable name to a different > executable via App Paths is not normal usage). > > Why not just have the second executable with a simple to type name > (easy_installw, say) and suggest people type *that* in the Run window? > Keep it simple! The main problem to be solved here is how do you put up a "Press enter to close window" prompt when a new console is opened up and not do that when running in an existing console? We could try to detect that condition. I looks like someone on the Cygwin XFree project mailing list solved that problem in 2003. I don't see this as having actually made it into their source base yet though. http://article.gmane.org/gmane.os.cygwin.xfree/5967 http://article.gmane.org/gmane.os.cygwin.xfree/5965 The second message has an UU encoded sample C implementation that could be modified for our purposes. -Chris From neveen.radwan at intel.com Wed Oct 4 21:05:26 2006 From: neveen.radwan at intel.com (Radwan, Neveen) Date: Wed, 4 Oct 2006 12:05:26 -0700 Subject: [Distutils] Python 2.4.3 Message-ID: <7F21D98CD0BB7246BA159696988A38A3D3DBA1@scsmsx413.amr.corp.intel.com> Hello, Can you let me know if there is a way to install this application silently on a system (ie. No dialog boxes) - are there any command line switches I can use? Thanks! Neveen Radwan Neveen Radwan Client Engineering California Engineering Computing (CAEC) - Santa Clara SC11-3-301 408.765.0446 - Office 408.765.7183 - Fax -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20061004/64f7447d/attachment.html From pje at telecommunity.com Wed Oct 4 21:19:24 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 04 Oct 2006 15:19:24 -0400 Subject: [Distutils] Transitioning a Python Library on a Shared Network Drive to eggs/setuptools In-Reply-To: <525f23e80610041108h79c4bbeema4224866767a31a3@mail.gmail.co m> Message-ID: <5.1.1.6.0.20061004150207.02ad1bc0@sparrow.telecommunity.com> At 02:08 PM 10/4/2006 -0400, Alexander Michael wrote: >In the past I've managed a shared library of Python packages by using >distutils to install them in secondary Library and Scripts directories on >a shared network drive. This worked fine, even in our multi-platform >environment. With advent of eggs, however, the secondary Library directory >must be a formal "Site Directory" and not just on sys.path. The extra >delay caused by the layer of network has caused simply getting --help for >a simple script to take almost three seconds when it previously only took >a tenth of a second. Some scripts that use many packages installed as eggs >on the network drive can take as many as 8 seconds just to display the >help message. > >I would like to install architecture independent Python packages in a >single shared location so that everyone using that location is >automatically upgraded. The in-house packages are modified about five >times a day on average. I would like to take advantage of setuptools >versioning (thus using the pkg_resources mechanisms) so deprecated >portions of the system can be kept intact in some frozen state of >development without having to include the version number in the package >name explicitly ( i.e. mymod, mymod2, .., mymod42). > >What is the recommended way of using eggs in such an environment? I'm not sure I understand your question. If you want to avoid the overhead due to network latency, you'd have to put the packages on a local drive. If you want to avoid the additional network round-trips being caused by setuptools looking for packages, you'll need to do away with the eggs (e.g. by installing everything using a package manager like RPM or bdist_wininst, etc.). I don't think there is any obvious way of accomplishing what you want without some way to "notice" that a newer version of something is available, yet without using the network. That seems to be a contradiction in terms. The closest thing I know of to what you're doing here is using "setup.py develop" on local revision-control checkouts of shared packages, but that requires that somebody explicitly update changed packages, or at least periodically run a script to do so. If I were in a situation like yours, I would arrange a revision control setup that allows all the subproject trees to be checked out under a common root, and a script to update each tree and rerun "setup.py develop" if any changes occurred, then leave it to the devs to decide when they want to sync. They could also *not* run "develop" (or not sync) packages they didn't want to. I find it hard to imagine using a networked filesystem to import Python code, however, even without eggs being involved, although I've heard rumors that Google does this. If you have to have a networked filesystem, however, I think you'll have to do without versioning, because it adds too many additional network round trips. The only thing I can think of that could work around this would be some sort of client-side caching of egg contents, so that startups can happen faster. From pje at telecommunity.com Wed Oct 4 21:27:37 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 04 Oct 2006 15:27:37 -0400 Subject: [Distutils] Draft of new setuptools installation instructions In-Reply-To: <4523917D.9030207@zope.com> References: <5.1.1.6.0.20061003170715.026db660@sparrow.telecommunity.com> <5.1.1.6.0.20061003170715.026db660@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20061004151946.027d3b10@sparrow.telecommunity.com> At 06:48 AM 10/4/2006 -0400, Jim Fulton wrote: >Phillip J. Eby wrote: >>I'm working on new installation instructions for setuptools 0.6c4, which >>will no longer use ez_setup.py for end-user manual installation of >>setuptools (as opposed to bundled installation with another package). > >This seems to me to be a pretty huge change for a c4 release. >Why not get 0.6 final out and do this for 0.7? Because I want 0.6 to be very stable; 0.7 may be a long time coming. The idea is to start making setuptools into *infrastructure*: something you just install like Python, not something that you need an automatic updater for. :) >>I'm looking for any feedback people might have, on the installation >>process itself or on the directions. Some particular areas of >>question/concern: >>* Should ez_setup perhaps launch a web browser pointing to these >>instructions, instead of trying to download setuptools itself? (This >>would bypass most of those annoying proxy problems right off the bat, at >>least when just installing setuptools.) > >Would it be possible, instead to provide a proxy option to >easy_install/ez_setup >(and the underlying setptools apis)? The HTTP_PROXY or http_proxy environment variable controls proxying for Python programs, including easy_install and ez_setup.py. What doesn't work is if they require NTLM authentication, which is not built into urllib. As long as the information is available in the Windows registry or environment, and NTLM auth isn't needed, it should work. But when NTLM auth is required, easy_install isn't going to work unless you use a proxy program like the one at http://ntlmaps.sf.net/ >>* The new install procedure bypasses firewall issues for installing >>setuptools itself, but doesn't do anything about the issue for packages >>that have dependencies. Is there anything else that can be done about that? > >Yes, provide a proxy option, ideally one that can be read from >a configuration file. That's probably not a bad idea, since it would be more discoverable than the environment variable and registry stuff. However, I'll have to tone up my urllib-fu in order to do that, as I have no idea how to set the proxy information from *code* (as opposed to the environment). It also won't help with ez_setup.py, which doesn't have any configuration file abilities or command-line option parsing of its own, although I could perhaps add the necessary arguments to the ez_setup API, since it's going away as a command-line tool anyway. From pje at telecommunity.com Wed Oct 4 22:15:38 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 04 Oct 2006 16:15:38 -0400 Subject: [Distutils] Draft of new setuptools installation instructions In-Reply-To: <79990c6b0610040123p6f360f38g2e786dbf9fd924ae@mail.gmail.co m> References: <5.1.1.6.0.20061003190616.026ec0d0@sparrow.telecommunity.com> <5.1.1.6.0.20061003170715.026db660@sparrow.telecommunity.com> <5.1.1.6.0.20061003174745.026c4ac8@sparrow.telecommunity.com> <5.1.1.6.0.20061003190616.026ec0d0@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20061004152804.026d26a8@sparrow.telecommunity.com> At 09:23 AM 10/4/2006 +0100, Paul Moore wrote: >On 10/4/06, Phillip J. Eby wrote: >>At 11:12 PM 10/3/2006 +0100, Paul Moore wrote: >> >Also, the automatic pause would annoy me. How would you make it *not* >> >happen if I ran the command from a console window I already had open? >> >>The idea would be to register a separate "easy_install-win.exe" under >>"easy_install" in App Paths. This version would pause after termination, >>while the normal version would not. Since console programs invoked via the >>App Paths mechanism *always* get their own new window, this should work out >>nicely. > >Sadly, not true. The replacement console program 4NT (which I use >pretty much exclusively) uses App Paths to locate executables. So if >you (in effect) alias easy_install to easy_install-win, you'll break >it for 4NT users. Ah crap. I wonder if there's any way to tell whether you're the only process on a particular console? That would fix this, I guess. There's a GetConsoleProcessList() API, but it doesn't even work on Win2K as far as I can tell; a minimum of XP is required. :( Further Googling shows that many other people with more Windows savvy than me have tried going this path and failed -- apparently the only way to find out who your parent process is is an undocumented function in ntdll. :( >Why not just have the second executable with a simple to type name >(easy_installw, say) and suggest people type *that* in the Run window? >Keep it simple! That doesn't solve the problem for other Python console scripts; It'd be ideal if I could solve this in the .exe launcher itself. From sjmachin at lexicon.net Wed Oct 4 22:49:13 2006 From: sjmachin at lexicon.net (John Machin) Date: Thu, 05 Oct 2006 06:49:13 +1000 Subject: [Distutils] Python 2.4.3 In-Reply-To: <7F21D98CD0BB7246BA159696988A38A3D3DBA1@scsmsx413.amr.corp.intel.com> References: <7F21D98CD0BB7246BA159696988A38A3D3DBA1@scsmsx413.amr.corp.intel.com> Message-ID: <45241E49.2080101@lexicon.net> On 5/10/2006 5:05 AM, Radwan, Neveen wrote: > > Can you let me know if there is a way to install this application > silently on a system (ie. No dialog boxes) ? are there any command line > switches I can use? > Hello, Neveen, This should answer your question: http://www.python.org/download/releases/2.4/msi/#automated If my guess as to which operating system you may be referring to is not correct, please give another hint :-) Cheers, John From pje at telecommunity.com Wed Oct 4 22:49:29 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 04 Oct 2006 16:49:29 -0400 Subject: [Distutils] More Windows questions In-Reply-To: <79990c6b0610040131x1c2bd88ud86fcca3d5cb88fa@mail.gmail.com > References: <5.1.1.6.0.20061003193456.02d20eb0@sparrow.telecommunity.com> <5.1.1.6.0.20061003193456.02d20eb0@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20061004161622.02c180e8@sparrow.telecommunity.com> At 09:31 AM 10/4/2006 +0100, Paul Moore wrote: >On 10/4/06, Phillip J. Eby wrote: >>Okay, I've looked at the registry a bit, and it seems like I can register >>App Paths either under HKLM (for the whole machine) or under HKCU (for the >>current user). However, both seem to have issues. >[...] >>2. Register HKCU "App Paths" entries for all console scripts, pointing to a >>second '-conio-.exe' wrapper for the same '-script.py', thus allowing >>console scripts to be run from the "Start/Run" command line. > >As I said in a previous email, keep away from App Paths, and in >particular, don't register an executable under any name but its own. > >There's very little documentation that I could find on what App Paths >should do, so all you've got to go on is experiment and looking at >what others do. No-one that I have seen ever registers an exe under a >different name. Actually, I've seen *two* on my own PC; one was a game, the other the Adobe SVG viewer. >And some programs (4NT, for example) expect the App >Path name to match the exe name (and give a broken user experience if >that isn't true). Broken how? From p.f.moore at gmail.com Wed Oct 4 23:46:26 2006 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 4 Oct 2006 22:46:26 +0100 Subject: [Distutils] Draft of new setuptools installation instructions In-Reply-To: <5.1.1.6.0.20061004152804.026d26a8@sparrow.telecommunity.com> References: <5.1.1.6.0.20061003170715.026db660@sparrow.telecommunity.com> <5.1.1.6.0.20061003174745.026c4ac8@sparrow.telecommunity.com> <5.1.1.6.0.20061003190616.026ec0d0@sparrow.telecommunity.com> <5.1.1.6.0.20061004152804.026d26a8@sparrow.telecommunity.com> Message-ID: <79990c6b0610041446v7ae625b6n6c43ce1b1cb87e47@mail.gmail.com> On 10/4/06, Phillip J. Eby wrote: > Ah crap. I wonder if there's any way to tell whether you're the only > process on a particular console? That would fix this, I guess. There's a > GetConsoleProcessList() API, but it doesn't even work on Win2K as far as I > can tell; a minimum of XP is required. :( > > Further Googling shows that many other people with more Windows savvy than > me have tried going this path and failed -- apparently the only way to find > out who your parent process is is an undocumented function in ntdll. :( Yes, I've seen a lot of people attempt to do this sort of thing, and they have always, in my experience, fallen short of their goals in some way. Ultimately, you're trying to fight a fundamental behaviour of the OS (the basic distinction between console-mode and GUI-mode executables). I've come to the conclusion long ago that you should always work *with* the system, rather than fighting it, whether that's Unix, Windows, or something else entirely. You can get very close sometimes (witness cygwin, for example) but you'll never paper over all the cracks. > >Why not just have the second executable with a simple to type name > >(easy_installw, say) and suggest people type *that* in the Run window? > >Keep it simple! > > That doesn't solve the problem for other Python console scripts; It'd be > ideal if I could solve this in the .exe launcher itself. You see this as a problem to be solved, I see it as a behaviour to be understood, accepted, and worked with :-) Paul. From p.f.moore at gmail.com Wed Oct 4 23:53:21 2006 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 4 Oct 2006 22:53:21 +0100 Subject: [Distutils] More Windows questions In-Reply-To: <5.1.1.6.0.20061004161622.02c180e8@sparrow.telecommunity.com> References: <5.1.1.6.0.20061003193456.02d20eb0@sparrow.telecommunity.com> <5.1.1.6.0.20061004161622.02c180e8@sparrow.telecommunity.com> Message-ID: <79990c6b0610041453x2f5691aclca14000f01ef400a@mail.gmail.com> > >And some programs (4NT, for example) expect the App > >Path name to match the exe name (and give a broken user experience if > >that isn't true). > > Broken how? The command xxx runs program yyy. Paul. From micktwomey at gmail.com Thu Oct 5 13:28:16 2006 From: micktwomey at gmail.com (Michael Twomey) Date: Thu, 5 Oct 2006 12:28:16 +0100 Subject: [Distutils] Building and using a custom python with buildout Message-ID: <50a522ca0610050428s67579a24x464c411e659ab215@mail.gmail.com> Hi, I've been experimenting with buildout, so far so good. One thing I'd like to do is build and use a python interpreter from source during the buildout process. I want to make my buildouts as self contained as possible, since I'm dealing with machines which might have python 2.3 installed but I want to use 2.4 or 2.5 for my app. Is this possible with buildout? I've built python but I can't figure out how to get buildout to use it: [buildout] develop = zc.recipe.cmmi parts = python mypython [python] recipe = zc.recipe.cmmi url = http://.../Python-2.5.tgz [mypython] recipe = zc.recipe.egg interpreter = mypython eggs = someegg I'm guessing something along the lines of a python var in the buildout section might work: python = ${buildout:parts-directory}/python/bin/python (this doesn't work). python = ${python:?}/bin/python (can't figure out a variable which works). cheers, Michael From pje at telecommunity.com Thu Oct 5 17:41:08 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 05 Oct 2006 11:41:08 -0400 Subject: [Distutils] Transitioning a Python Library on a Shared Network Drive to eggs/setuptools In-Reply-To: <525f23e80610050759p7d8bc504o1fad10350db23b57@mail.gmail.co m> References: <5.1.1.6.0.20061004150207.02ad1bc0@sparrow.telecommunity.com> <5.1.1.6.0.20061004150207.02ad1bc0@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20061005112509.026f4b90@sparrow.telecommunity.com> At 10:59 AM 10/5/2006 -0400, Alexander Michael wrote: >I want to avoid the additional network round-trips being caused by setuptools >looking for packages without throwing out the proverbial baby with the >bath water (i.e. ditching eggs entirely). > >Looking into it further, it appears that a fair portion of the overhead is >incurred by making the network directory a sitedir (with site.addsitedir). >The following alternate setup seems to work a little better. In a .pth >file in the local site-packages directory I read the list of egg pathnames >from the network drive and add these eggs to sys.path myself. So, you're doing site.addpackage() to load the easy-install.pth that's on the network drive? That makes sense. > That way, I don't need to make the remote packages directory a sitedir > just so the easy-install.pth can be read from it. This allows me to > control the available versions and eggs remotely, while minimizing > network access. I am willing to pay the cost of reading each package from > the network directory (as well as the package list) in order to achieve > transparent updating. Now getting a simple help message is three times > faster and almost tolerable. > >Nevertheless, it sounds like I will either need to cache the shared >library on each users computer or ditch eggs altogether in order to bring >performance back to acceptable levels. >Since caching could make performance even better than before, I will try >to set this up. If you're going to read the easy-install.pth from the network drive, you could actually take one more step and see whether any eggs are listed there that weren't before, and go ahead and copy them to the local machine. However, there's another possibility regarding what's happening that you might consider. If your original setup was installing eggs to an easy-install.pth, you should try installing eggs to the network drive in --multi-version mode, so that only programs that explicitly request those eggs will add them to sys.path. The site directory will only get read once that way, and Python won't try to read the zip directories of every single egg, which is probably what's happening now. For this to work, your scripts must be wrappers generated by setuptools, or you must explicitly use pkg_resources.require() to ask for the libraries you need. (Recursive dependency lookups are automatic, however.) I'd suggest you give this a try, as it's an out-of-the-box configuration but one that's likely to get closer to your previous performance or *maybe* even exceed it due to its more effective use of zip files. Here's what you should do: the code you now have reading easy-install.pth from the network, should just tack the directory on the *end* of sys.path. Ignore easy-install.pth altogether, and in fact you can remove it from the network drive, and in future use the -m argument to easy_install when putting eggs on the network drive, so that it doesn't put anything in the .pth file. Get your scripts to request dependencies explicitly, and you should then have the maximum possible performance for an egg-based setup, because the directory will be listed only once, and zip directories will only be read for the actually required eggs. >If I do decide to ditch eggs altogether, if someone gives me an egg, is >there a way I can "unpack" it as if I did a traditional distutils install? Yes; simply extract it, and rename the resulting EGG-INFO directory to originalname.egg-info/, where originalname is the name of the original egg file. This will give you a "single version, externally managed" egg, in the format that is used for RPM, bdist_wininst, and other "system packager" egg installs. >The context is a scientific data analysis environment in which a group of >user-developers (nearly everyone works in both roles) both write data >analysis tools and perform data analysis. The tools are ever and rapidly >evolving along with the analysis, so the transparent upgrading that occurs >by using a shared drive has been convenient. We work in individual SVN >checkouts. After testing and committing our changes, we install the update >to the shared drive where by everyone automatically gets the change and we >assure that everyone is in sync. And I suppose it's asking too much to run "setup.py develop" on the SVN checkout when you want to get updated versions? (Because you could configure it to copy eggs down from the network drive at the time, using "setup.py develop -af /path/to/eggs".) Just a thought, but I suppose if you just want the *tools* to be up to date whenever you run them... I'm just confused by the idea that in your shop, if I ran an analysis twice in a row without taking any special action, I might end up with different results. But oh well. From lxander.m at gmail.com Thu Oct 5 22:01:18 2006 From: lxander.m at gmail.com (Alexander Michael) Date: Thu, 5 Oct 2006 16:01:18 -0400 Subject: [Distutils] Transitioning a Python Library on a Shared Network Drive to eggs/setuptools In-Reply-To: <5.1.1.6.0.20061005112509.026f4b90@sparrow.telecommunity.com> References: <5.1.1.6.0.20061004150207.02ad1bc0@sparrow.telecommunity.com> <5.1.1.6.0.20061005112509.026f4b90@sparrow.telecommunity.com> Message-ID: <525f23e80610051301s728a657cre5b6e2b33cf02e93@mail.gmail.com> On 10/5/06, Phillip J. Eby wrote: > > At 10:59 AM 10/5/2006 -0400, Alexander Michael wrote: > >I want to avoid the additional network round-trips being caused by > setuptools > >looking for packages without throwing out the proverbial baby with the > >bath water (i.e. ditching eggs entirely). > > > >Looking into it further, it appears that a fair portion of the overhead > is > >incurred by making the network directory a sitedir (with site.addsitedir > ). > >The following alternate setup seems to work a little better. In a .pth > >file in the local site-packages directory I read the list of egg > pathnames > >from the network drive and add these eggs to sys.path myself. > > So, you're doing site.addpackage() to load the easy-install.pth that's on > the network drive? That makes sense. > > > > That way, I don't need to make the remote packages directory a sitedir > > just so the easy-install.pth can be read from it. This allows me to > > control the available versions and eggs remotely, while minimizing > > network access. I am willing to pay the cost of reading each package > from > > the network directory (as well as the package list) in order to achieve > > transparent updating. Now getting a simple help message is three times > > faster and almost tolerable. > > > >Nevertheless, it sounds like I will either need to cache the shared > >library on each users computer or ditch eggs altogether in order to bring > >performance back to acceptable levels. > >Since caching could make performance even better than before, I will try > >to set this up. > > If you're going to read the easy-install.pth from the network drive, you > could actually take one more step and see whether any eggs are listed > there > that weren't before, and go ahead and copy them to the local machine. > > However, there's another possibility regarding what's happening that you > might consider. If your original setup was installing eggs to an > easy-install.pth, you should try installing eggs to the network drive in > --multi-version mode, so that only programs that explicitly request those > eggs will add them to sys.path. The site directory will only get read > once > that way, and Python won't try to read the zip directories of every single > egg, which is probably what's happening now. > > For this to work, your scripts must be wrappers generated by setuptools, > or > you must explicitly use pkg_resources.require() to ask for the libraries > you need. (Recursive dependency lookups are automatic, however.) > > I'd suggest you give this a try, as it's an out-of-the-box configuration > but one that's likely to get closer to your previous performance or > *maybe* > even exceed it due to its more effective use of zip files. > > Here's what you should do: the code you now have reading easy-install.pth > from the network, should just tack the directory on the *end* of > sys.path. Ignore easy-install.pth altogether, and in fact you can remove > it from the network drive, and in future use the -m argument to > easy_install when putting eggs on the network drive, so that it doesn't > put > anything in the .pth file. Get your scripts to request dependencies > explicitly, and you should then have the maximum possible performance for > an egg-based setup, because the directory will be listed only once, and > zip > directories will only be read for the actually required eggs. > > > >If I do decide to ditch eggs altogether, if someone gives me an egg, is > >there a way I can "unpack" it as if I did a traditional distutils > install? > > Yes; simply extract it, and rename the resulting EGG-INFO directory to > originalname.egg-info/, where originalname is the name of the original egg > file. This will give you a "single version, externally managed" egg, in > the format that is used for RPM, bdist_wininst, and other "system > packager" > egg installs. > > > >The context is a scientific data analysis environment in which a group of > >user-developers (nearly everyone works in both roles) both write data > >analysis tools and perform data analysis. The tools are ever and rapidly > >evolving along with the analysis, so the transparent upgrading that > occurs > >by using a shared drive has been convenient. We work in individual SVN > >checkouts. After testing and committing our changes, we install the > update > >to the shared drive where by everyone automatically gets the change and > we > >assure that everyone is in sync. > > And I suppose it's asking too much to run "setup.py develop" on the SVN > checkout when you want to get updated versions? (Because you could > configure it to copy eggs down from the network drive at the time, using > "setup.py develop -af /path/to/eggs".) Just a thought, but I suppose if > you just want the *tools* to be up to date whenever you run them... I'm > just confused by the idea that in your shop, if I ran an analysis twice in > a row without taking any special action, I might end up with different > results. But oh well. > This discussion has given me some tangible options to choose between: 1) Keep the packages installed in the remote directory: a) with eggs placed in sys.path manually, b) installed locally from remote source directories with setup.pydevelop, c) eggs installed with --multi-version and accessed via pkg_resources.require d) as single version, externally managed eggs, I have tested all but the last option and found them to be of the similar speed and faster than making the remote directory a sitedir. 2) Keep local computer installation updated from: a) remote directory of eggs i) with update by user run script ii) with update triggered by .pth magic b) source SVN checkout directory updated and installed by user initiated script I will test some of these options next. I really appreciate the help. Thanks Phillip! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20061005/2d16ba64/attachment.html From venkatbo at yahoo.com Sun Oct 8 03:57:12 2006 From: venkatbo at yahoo.com (Venkat Bommakanti) Date: Sat, 7 Oct 2006 18:57:12 -0700 (PDT) Subject: [Distutils] cross compiling using setuptools Message-ID: <20061008015712.32144.qmail@web30511.mail.mud.yahoo.com> Hi folks, I have a need to cross compile (i686 linux build system -> ppc linux target system) some py/C source pkgs: . RuleDispatch-0.5a0.dev-r2115.tar.gz . cElementTree-1.0.5-20051216.zip . PyProtocols-1.0a0dev_r2082.zip . Cheetah-2.0rc7.tar.gz to generate target-specific eggs... Actually, all are a part of TurboGears (TG). I was able to download all the comps to a local directory (i686 linux with python 2.4.2 compiled from sources, w/zlib enabled). Now, . while in the download dir (curr-dir - .), . with PYTHONPATH that includes "/usr/lib/python2.4/site-packages", and . with a command like: "easy_install --prefix=/usr -s /usr/bin -f . TurboGears" I'm able to build the native i686 target and deploy the i686 eggs to : /usr/lib/python2.4/site-packages just fine... No problem there. Assuming I can use setuptools as-is and "easy_install ..." invoke syntax like above - for the cross build (ppc target), how would I specify the options meant for: . setup.py (of each of the above comps): . -c (cross-gcc) . bdest_egg (setuptools): . --dist-dir . --plat-name . exclude-source in the "easy_install ..." command. I read: http://peak.telecommunity.com/DevCenter/setuptools#bdist-egg-create-a-python-egg-for-the-project but its unclear how the above options are passed on to "easy_install ...". I realize that I'd have to use a PYTHONPATH that includes something like: "/usr/lib/python2.4/site-packages" for the ppc-target, so it doesn't complain about the: "bad install directory or PYTHONPATH" error... Also, 'am assuming its not a problem to include a path to ppc-eggs in the PYTHONPATH setting, before invoking the "easy_install ..." command for the cross compile. As an aside, would also like to know how I could enable the creation of target-specific .pyc files, in such a cross-compile scenario. I was able to build python itself in a cross compile (i686 build system -> ppc target system) mode, but the .pyc it generates appear to be i686-specific ones and not ppc-ones... when deployed to the ppc-system, it complains about the "bad magic number" error... Much appreciate any help. Thanks. /venkat From pje at telecommunity.com Sun Oct 8 06:39:00 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 08 Oct 2006 00:39:00 -0400 Subject: [Distutils] cross compiling using setuptools In-Reply-To: <20061008015712.32144.qmail@web30511.mail.mud.yahoo.com> Message-ID: <5.1.1.6.0.20061008003532.0274c490@sparrow.telecommunity.com> At 06:57 PM 10/7/2006 -0700, Venkat Bommakanti wrote: >Assuming I can use setuptools as-is and "easy_install ..." invoke syntax >like above - for the cross build (ppc target), how would I specify the >options meant for: > . setup.py (of each of the above comps): > > . -c (cross-gcc) > . bdest_egg (setuptools): > . --dist-dir > . --plat-name > . exclude-source >in the "easy_install ..." command. I read: > >http://peak.telecommunity.com/DevCenter/setuptools#bdist-egg-create-a-python-egg-for-the-project >but its unclear how the above options are passed on to "easy_install ...". Put them in your ~/.pydistutils.cfg file, using e.g. [build] compiler = ... [bdist_egg] plat_name = ... exclude_source = 1 >I realize that I'd have to use a PYTHONPATH that includes something like: > "/usr/lib/python2.4/site-packages" >for the ppc-target, so it doesn't complain about the: > "bad install directory or PYTHONPATH" Use the -m option to easy_install, and it won't complain about that. >error... Also, 'am assuming its not a problem to include a path to >ppc-eggs in the PYTHONPATH setting, before invoking the "easy_install ..." >command for the cross compile. > >As an aside, would also like to know how I could enable the creation of >target-specific .pyc files, in such a cross-compile scenario. I was able >to build python itself in a cross compile (i686 build system -> ppc target >system) mode, but the .pyc it generates appear to be i686-specific ones >and not ppc-ones... when deployed to the ppc-system, it complains about >the "bad magic number" error... Odd; as far as I know, .pyc/.pyo files are completely platform independent. They are specific to Python version, however, so both machines would have to be e.g. Python 2.3, or both 2.4. From venkatbo at yahoo.com Sun Oct 8 23:07:46 2006 From: venkatbo at yahoo.com (Venkat Bommakanti) Date: Sun, 8 Oct 2006 14:07:46 -0700 (PDT) Subject: [Distutils] cross compiling using setuptools Message-ID: <20061008210746.23578.qmail@web30505.mail.mud.yahoo.com> Thanks Phillip. Have a followup question below. /venkat >>Assuming I can use setuptools as-is and "easy_install ..." invoke syntax >>like above - for the cross build (ppc target), how would I specify the >>options meant for: >> . setup.py (of each of the above comps): >> >> . -c (cross-gcc) >> . bdest_egg (setuptools): >> . --dist-dir >> . --plat-name >> . exclude-source >>in the "easy_install ..." command. I read: >> >>http://peak.telecommunity.com/DevCenter/setuptools#bdist-egg-create-a-python-egg-for-the-project >>but its unclear how the above options are passed on to "easy_install ...". >Put them in your ~/.pydistutils.cfg file, using e.g. >[build] >compiler = ... >[bdist_egg] >plat_name = ... >exclude_source = 1 I'm trying to create just one makefile, that will take the as a param, so it can be used to build both i686 and ppc targets of TG (in this case). So, how would I specify that variance in just **one** ~/.pydistutils.cfg OR is there a way to have setuptools pickup different versions of this file, based on the chosen for the makefile. Basically, I'll need something like: # i686 specific section [build] compiler = [bdist_egg] plat_name = i686 exclude_source = 1 and, # ppc specific section [build] compiler = [bdist_egg] plat_name = ppc exclude_source = 1 Thanks, /venkat From pje at telecommunity.com Sun Oct 8 23:17:32 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 08 Oct 2006 17:17:32 -0400 Subject: [Distutils] cross compiling using setuptools In-Reply-To: <20061008210746.23578.qmail@web30505.mail.mud.yahoo.com> Message-ID: <5.1.1.6.0.20061008171641.027284f0@sparrow.telecommunity.com> At 02:07 PM 10/8/2006 -0700, Venkat Bommakanti wrote: >So, how would I specify that variance in just **one** ~/.pydistutils.cfg >OR is there a way to have setuptools pickup different versions of this >file, based on the chosen for the makefile. You could use different values of $HOME, since that's where the per-user config file is read from. From venkatbo at yahoo.com Sun Oct 8 23:24:59 2006 From: venkatbo at yahoo.com (Venkat Bommakanti) Date: Sun, 8 Oct 2006 14:24:59 -0700 (PDT) Subject: [Distutils] cross compiling using setuptools Message-ID: <20061008212459.68351.qmail@web30503.mail.mud.yahoo.com> >At 02:07 PM 10/8/2006 -0700, Venkat Bommakanti wrote: >>So, how would I specify that variance in just **one** ~/.pydistutils.cfg >>OR is there a way to have setuptools pickup different versions of this >>file, based on the chosen for the makefile. >You could use different values of $HOME, since that's where the per-user >config file is read from. Wondering if a "setup.cfg" for this component (TG) in the component's directory itself, will serve the purpose. That ways, each developer wouldn't have maintain thier own ~/.pydistutils.cfg files. Thanks again, /venkat From pje at telecommunity.com Sun Oct 8 23:35:20 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 08 Oct 2006 17:35:20 -0400 Subject: [Distutils] cross compiling using setuptools In-Reply-To: <20061008212459.68351.qmail@web30503.mail.mud.yahoo.com> Message-ID: <5.1.1.6.0.20061008173325.02728190@sparrow.telecommunity.com> At 02:24 PM 10/8/2006 -0700, Venkat Bommakanti wrote: > Wondering if a "setup.cfg" for this component (TG) in the component's > directory itself, will serve the purpose. Well, if each project is going to be built for only one platform, sure. But the setup.cfg *overrides* the .pydistutils.cfg, so you don't want to distribute a bunch of site/platform-specific options in the project's published setup.cfg. In any case, I thought your original issue had to do with getting easy_install to handle these packages automatically, without modifying the original packages in any way. And that's the context in which I've been answering your questions. From venkatbo at yahoo.com Sun Oct 8 23:54:31 2006 From: venkatbo at yahoo.com (Venkat Bommakanti) Date: Sun, 8 Oct 2006 14:54:31 -0700 (PDT) Subject: [Distutils] cross compiling using setuptools Message-ID: <20061008215431.73468.qmail@web30504.mail.mud.yahoo.com> >At 02:24 PM 10/8/2006 -0700, Venkat Bommakanti wrote: >> Wondering if a "setup.cfg" for this component (TG) in the component's >> directory itself, will serve the purpose. >Well, if each project is going to be built for only one platform, >sure. But the setup.cfg *overrides* the .pydistutils.cfg, so you don't >want to distribute a bunch of site/platform-specific options in the >project's published setup.cfg. >In any case, I thought your original issue had to do with getting >easy_install to handle these packages automatically, without modifying the >original packages in any way. And that's the context in which I've been >answering your questions. I apologize for the confusion here... Will try to clarify the full need... Basically, the need is such: . a developer would snap a tree . build python , TG for a of choice (i686/ppc) A developer could potentially fire the build of both targets simultaneously. And to build the ppc python/TG (cross target) they'd use the necessary i686 python that is part of a directory on the i686 build box. Under those circumstances, and also the desire to keep all build related files (incuding .pydistutils.cfg or equivalents) within the source tree itself, I wanted to know the best recourse. Using $HOME/.pydistutils.cfg may not be a viable option due the fact that it'd reside outside the source tree, and could become a maintenance issue... Would like a solution where one gets the cfg file (or equivalent) when they snap a tree direct... I was hoping something like: /TurboGears/ with info relevant to: [build] compiler = $COMPILER (native or cross-gcc) [bdist_egg] plat_name = $TARGET (i686 or ppc) exclude_source = 1 would let me specify (target-based) info that is picked up according to the build target. Thanks again, /venkat From pje at telecommunity.com Mon Oct 9 00:21:04 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 08 Oct 2006 18:21:04 -0400 Subject: [Distutils] cross compiling using setuptools In-Reply-To: <20061008215431.73468.qmail@web30504.mail.mud.yahoo.com> Message-ID: <5.1.1.6.0.20061008181909.029b53f0@sparrow.telecommunity.com> At 02:54 PM 10/8/2006 -0700, Venkat Bommakanti wrote: >Using $HOME/.pydistutils.cfg may not be a viable option due the fact >that it'd reside outside the source tree, and could become a >maintenance issue... Would like a solution where one gets the cfg file >(or equivalent) when they snap a tree direct... > >I was hoping something like: > /TurboGears/ >with info relevant to: > > [build] > compiler = $COMPILER (native or cross-gcc) > > [bdist_egg] > plat_name = $TARGET (i686 or ppc) > exclude_source = 1 > >would let me specify (target-based) info that is picked up according >to the build target. You could certainly put some code in setup.py that *generates* a setup.cfg based on this kind of information, if it's found in the environment, sure. From venkatbo at yahoo.com Mon Oct 9 17:39:05 2006 From: venkatbo at yahoo.com (Venkat Bommakanti) Date: Mon, 9 Oct 2006 08:39:05 -0700 (PDT) Subject: [Distutils] cross compiling using setuptools Message-ID: <20061009153905.14180.qmail@web30514.mail.mud.yahoo.com> >>Assuming I can use setuptools as-is and "easy_install ..." invoke syntax >>like above - for the cross build (ppc target), how would I specify the >>options meant for: >> . setup.py (of each of the above comps): >> . -c (cross-gcc) >> . bdest_egg (setuptools): >> . --dist-dir >> . --plat-name >> . exclude-source >>in the "easy_install ..." command. I read: >Put them in your ~/.pydistutils.cfg file, using e.g. >[build] >compiler = ... >[bdist_egg] >plat_name = ... >exclude_source = 1 Based on this suggestion, for the cross compile case, I tried to set: [bdist_egg] plat_name = ppc but, it still insists on falling back to i686 for plat_name. Could be I'm doing something fundamentally wrong. 'am also trying to "set" as many params to easy_install / setup.py-of-comp (in TG's suite of components) as possible this way... but its unclear if I'm doing them right... Is there a reference somewhere in (http://peak.telecommunity.com/) to all . the sections that can go into these files, and . the key/value pairs of settings, that can go into each section. For example, I couldn't see anywhere it documented that [bdist_egg] is one of the many valid sections in these files. I'm not doubting the earlier statement, but just unable to find a single place that lists all the possible cases. 'am also assuming the syntax is the same for setup.cfg (local to TG source dir) and pydistutils.cfg ($HOME). Thanks, /venkat From pje at telecommunity.com Mon Oct 9 18:10:56 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 09 Oct 2006 12:10:56 -0400 Subject: [Distutils] cross compiling using setuptools In-Reply-To: <20061009153905.14180.qmail@web30514.mail.mud.yahoo.com> Message-ID: <5.1.1.6.0.20061009120552.04676ff8@sparrow.telecommunity.com> At 08:39 AM 10/9/2006 -0700, Venkat Bommakanti wrote: >Based on this suggestion, for the cross compile case, I tried to set: > [bdist_egg] > plat_name = ppc >but, it still insists on falling back to i686 for plat_name. Could be I'm >doing something fundamentally wrong. Dunno. Is this in $HOME/.pydistutils.cfg? >'am also trying to "set" as many params to easy_install / setup.py-of-comp >(in TG's suite of components) as possible this way... but its unclear if >I'm doing them right... > >Is there a reference somewhere in (http://peak.telecommunity.com/) to all > . the sections that can go into these files, and > . the key/value pairs of settings, that can go into each section. These are standard distutils configuration files. The way it works is that any distutils "command" can have a section, and any command-line option can have a value set for it. You use the full option name, change any '-' in it to '_', and then use that as the setting name in the section. Boolean options are set to 1 or 0, others are just text after the '='. So, the reference to these is "setup.py commandname --help" (e.g. "bdist_egg --help") or the reference manual for setuptools and its extra commands. >For example, I couldn't see anywhere it documented that [bdist_egg] is one >of the many valid sections in these files. I'm not doubting the earlier >statement, >but just unable to find a single place that lists all the possible cases. That's because the config files just list command-line options, and they basically establish defaults that can still be overridden on the command line. > 'am also >assuming the syntax is the same for setup.cfg (local to TG source dir) and >pydistutils.cfg ($HOME). On Unix-like platforms, it's ".pydistutils.cfg" (note the '.' at the beginning), NOT "pydistutils.cfg". The latter name is used only on Windows. From venkatbo at yahoo.com Mon Oct 9 18:56:33 2006 From: venkatbo at yahoo.com (Venkat Bommakanti) Date: Mon, 9 Oct 2006 09:56:33 -0700 (PDT) Subject: [Distutils] cross compiling using setuptools Message-ID: <20061009165634.35283.qmail@web30511.mail.mud.yahoo.com> >>Based on this suggestion, for the cross compile case, I tried to set: >> [bdist_egg] >> plat_name = ppc >>but, it still insists on falling back to i686 for plat_name. Could be I'm >>doing something fundamentally wrong. >Dunno. Is this in $HOME/.pydistutils.cfg? I'm actually using a seperate for each . So, they are: /temp_i686/setup.cfg and /temp_ppc/setup.cfg They have their own (temp) set of TG comps in their native tar/zip formats. This will allow for a developer to snap a tree and build from there, without any build-specifc files in their $HOMEs. Each will have /temp_/easy_install -f . TurboGears invoked from their respective directories, and will use their respective setup.cfg files. >> 'am also >>assuming the syntax is the same for setup.cfg (local to TG source dir) and >>pydistutils.cfg ($HOME). >On Unix-like platforms, it's ".pydistutils.cfg" (note the '.' at the >beginning), NOT "pydistutils.cfg". The latter name is used only on Windows. Based on the above layout, and from what I've understood so far, I'm assuming: /temp_/setup.cfg is the equivalent of: $HOME/.pydistutils.cfg per the responses so far, and notes in pages like: http://docs.python.org/inst/config-syntax.html Is is accurate to say that the contents of setup.cfg & pydistutils.cfg are the exactly the same ? Thanks a lot, /venkat From pje at telecommunity.com Mon Oct 9 20:34:14 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 09 Oct 2006 14:34:14 -0400 Subject: [Distutils] cross compiling using setuptools In-Reply-To: <20061009165634.35283.qmail@web30511.mail.mud.yahoo.com> Message-ID: <5.1.1.6.0.20061009143340.0272f238@sparrow.telecommunity.com> At 09:56 AM 10/9/2006 -0700, Venkat Bommakanti wrote: >Based on the above layout, and from what I've understood so far, >I'm assuming: > /temp_/setup.cfg >is the equivalent of: > $HOME/.pydistutils.cfg Only if the setup.py being run is in the same directory as the setup.cfg. From venkatbo at yahoo.com Tue Oct 10 01:23:06 2006 From: venkatbo at yahoo.com (Venkat Bommakanti) Date: Mon, 9 Oct 2006 16:23:06 -0700 (PDT) Subject: [Distutils] cross compiling using setuptools Message-ID: <20061009232306.40982.qmail@web30504.mail.mud.yahoo.com> >>Based on this suggestion, for the cross compile case, I tried to set: >> [bdist_egg] >> plat_name = ppc >>but, it still insists on falling back to i686 for plat_name. Could be I'm >>doing something fundamentally wrong. >Dunno. Is this in $HOME/.pydistutils.cfg? For the cross compile case (i686 build box -> ppc target), rather than the setup.py/setup.cfg option in the TurboGears src/pkg dir, I thought I'll try the basic method of using $HOME/.pydistutils.cfg first. So I created one ($HOME/.pydistutils.cfg) with the following contents: [build] compiler=powerpc-unknown-linux-gcc [bdist_egg] pla_name=ppc When I run: easy_install --prefix=usr -s usr/bin -f . TG.egg I get an error msg: error: Setup script exited with error: don't know how to compile C/C++ code on platform 'posix' with 'powerpc-unknown-linux-gcc' compiler I have 'powerpc-unknown-linux-gcc' in my path. Could it be that its not picking up some include dirs etc. I also tried the same with i686 build box -> i686 target, pointing to my i686 gcc 'i686-pc-linux-gcc'... [build] compiler=i686-pc-linux-gcc [bdist_egg] pla_name=i686 I get the same error... error: Setup script exited with error: don't know how to compile C/C++ code on platform 'posix' with 'i686-pc-linux-gcc' compiler Any idea what may be wrong here... Thanks much, /venkat From pje at telecommunity.com Tue Oct 10 01:55:01 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 09 Oct 2006 19:55:01 -0400 Subject: [Distutils] cross compiling using setuptools In-Reply-To: <20061009232306.40982.qmail@web30504.mail.mud.yahoo.com> Message-ID: <5.1.1.6.0.20061009194529.028b2968@sparrow.telecommunity.com> At 04:23 PM 10/9/2006 -0700, Venkat Bommakanti wrote: >For the cross compile case (i686 build box -> ppc target), rather than >the setup.py/setup.cfg option in the TurboGears src/pkg dir, I thought >I'll try the basic method of using $HOME/.pydistutils.cfg first. So I >created one ($HOME/.pydistutils.cfg) with the following contents: > [build] > compiler=powerpc-unknown-linux-gcc > [bdist_egg] > pla_name=ppc > >When I run: > easy_install --prefix=usr -s usr/bin -f . TG.egg >I get an error msg: > error: Setup script exited with error: don't know how to compile C/C++ > code on platform 'posix' with 'powerpc-unknown-linux-gcc' > compiler >I have 'powerpc-unknown-linux-gcc' in my path. Could it be that its not >picking up some include dirs etc. $ python setup.py build_ext --help ... Options for 'build_ext' command: ... --compiler (-c) specify the compiler type ... --help-compiler list available compilers $ python setup.py build_ext --help-compiler List of available compilers: --compiler=bcpp Borland C++ Compiler --compiler=cygwin Cygwin port of GNU C Compiler for Win32 --compiler=emx EMX port of GNU C Compiler for OS/2 --compiler=mingw32 Mingw32 port of GNU C Compiler for Win32 --compiler=msvc Microsoft Visual C++ --compiler=mwerks MetroWerks CodeWarrior --compiler=unix standard UNIX-style compiler These are the only options the distutils provide for --compiler, and setuptools doesn't currently offer any way to extend this. (Although if somebody wanted to contribute a mechanism to do so based on entry points, that would be way cool.) From ianb at colorstudy.com Tue Oct 10 04:36:40 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Mon, 09 Oct 2006 21:36:40 -0500 Subject: [Distutils] easy_install --always-copy and setup.py develop Message-ID: <452B0738.7060006@colorstudy.com> I have a package (PasteDeploy) installed with "python setup.py develop", at version 0.9.7dev-r5544. There's a released version 0.9.6. When I do "easy_install --always-copy --install-dir /some/dir SomePackageThatRequiresPasteDeploy" it gives me errors like error: Could not find suitable distribution for Requirement.parse('PasteDeploy==0.9.7dev-r5544') (--always-copy skips system and development eggs) There is no requirement like that, it seems to be coming up with that version specifier on its own. If I try to install PasteDeploy specifically it works fine. If I remove that egg-link from develop it works fine. (Note: my end goal is to create a bundle of all the eggs that an app needs to run) -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From venkatbo at yahoo.com Tue Oct 10 06:05:47 2006 From: venkatbo at yahoo.com (Venkat Bommakanti) Date: Mon, 9 Oct 2006 21:05:47 -0700 (PDT) Subject: [Distutils] compiler selection with bdist_egg of setuptools Message-ID: <20061010040547.82578.qmail@web30503.mail.mud.yahoo.com> Hi, While compiling some py/C source pkgs, such as: . RuleDispatch-0.5a0.dev-r2115.tar.gz . cElementTree-1.0.5-20051216.zip . PyProtocols-1.0a0dev_r2082.zip . Cheetah-2.0rc7.tar.gz I noticed in the logs that each (after being unpacked) has the following command invoked: setup.py -q bdist_egg --dist-dir /tmp/.... Since selecting a 'compiler' is not an option for the "bdist_egg" command, are even the settings in ~/.pydistutils.cfg file: [build] compiler = unix [bdist_egg] plat_name = ppc picked up ? They appear not to be picked up, and I'm suspecting that since the [build] settings are valid only for the "build" command, it will not have any effect on the "bdist_egg" command used by all the above comps. Is that accurate ? If plat_name is not picked up, how is setuptools figuring out that the platform is i686 (given that the build is being done for a i686 linux box). Verbose logs don't show where/how that decision is made... Thanks, /venkat From pje at telecommunity.com Tue Oct 10 06:21:30 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 10 Oct 2006 00:21:30 -0400 Subject: [Distutils] compiler selection with bdist_egg of setuptools In-Reply-To: <20061010040547.82578.qmail@web30503.mail.mud.yahoo.com> Message-ID: <5.1.1.6.0.20061010001859.02736b80@sparrow.telecommunity.com> At 09:05 PM 10/9/2006 -0700, Venkat Bommakanti wrote: >Hi, > >While compiling some py/C source pkgs, such as: > . RuleDispatch-0.5a0.dev-r2115.tar.gz > . cElementTree-1.0.5-20051216.zip > . PyProtocols-1.0a0dev_r2082.zip > . Cheetah-2.0rc7.tar.gz >I noticed in the logs that each (after being unpacked) has >the following command invoked: > setup.py -q bdist_egg --dist-dir /tmp/.... > >Since selecting a 'compiler' is not an option for the >"bdist_egg" command, are even the settings in >~/.pydistutils.cfg file: > [build] > compiler = unix > [bdist_egg] > plat_name = ppc >picked up ? > >They appear not to be picked up, and I'm suspecting that >since the [build] settings are valid only for the "build" command, >it will not have any effect on the "bdist_egg" command used >by all the above comps. Is that accurate ? No, it's not accurate. bdist_egg invokes various build commands (such as build_ext) which get their default --compiler setting from the [build] configuration. >If plat_name is not picked up, how is setuptools figuring >out that the platform is i686 (given that the build is being >done for a i686 linux box). Verbose logs don't show where/how >that decision is made... Try running with $DISTUTILS_DEBUG set to "yes". You will get a ton of output showing what configuration files are being parsed and how the options for each command are getting set. Look at what's happening when the second-level setup is run (i.e., the package being asked to 'bdist_egg'); the first-level (easy_install itself) won't make any difference. From pierre at saiph.com Tue Oct 10 16:29:48 2006 From: pierre at saiph.com (Pierre Imbaud) Date: Tue, 10 Oct 2006 10:29:48 -0400 Subject: [Distutils] RuleDispatch In-Reply-To: References: Message-ID: Anthony Tarlano wrote: > Hi all.. > > Firstly, I would like to say thanks for the setuptools package, which > I was introduced to after reading about the RuleDispatch package on > the IBM developerworks charming python series. Oh btw. RuleDispatch is > the most useful python package that I have seen in the last year in > the python world, so thanks for that also ;) I read this posting, made me curious about RuleDispatch, but I cant find a hint about this package! Is it dead, or just in artificial sleep? > .... From pje at telecommunity.com Tue Oct 10 17:05:10 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 10 Oct 2006 11:05:10 -0400 Subject: [Distutils] RuleDispatch In-Reply-To: References: Message-ID: <5.1.1.6.0.20061010110012.03019d98@sparrow.telecommunity.com> At 10:29 AM 10/10/2006 -0400, Pierre Imbaud wrote: >Anthony Tarlano wrote: > > > Hi all.. > > > > Firstly, I would like to say thanks for the setuptools package, which > > I was introduced to after reading about the RuleDispatch package on > > the IBM developerworks charming python series. Oh btw. RuleDispatch is > > the most useful python package that I have seen in the last year in > > the python world, so thanks for that also ;) >I read this posting, made me curious about RuleDispatch, but I cant find >a hint about this package! Is it dead, or just in artificial sleep? > > .... See http://www.eby-sarna.com/pipermail/peak/2006-May/002534.html for the details. RuleDispatch is on life support: it is available only via SVN and I'm doing only bug fixes. Python 2.5 is not currently supported, in part due to grammar changes that affect RuleDispatch's parser and in part due to the fact that Pyrex does not support Python 2.5's new-style exceptions yet. I've also just released 'simplegeneric', a super-simple single-dispatch generic function implementation in 100 lines of Python, with no dependencies (except that it uses setuptools for installation). An in-development version of PEAK-Rules is also available via SVN; it does not support predicate dispatch yet, but so far does support multiple dispatch and method combining. PEAK-Rules has a much better test suite than RuleDispatch, as it has been test-first developed from the ground up. (As was 'simplegeneric'.) RuleDispatch was first written before I was truly "test infected". Anyway, this subject is off-topic for distutils-sig, though, so please send any reply via the PEAK mailing list. From venkatbo at yahoo.com Tue Oct 10 20:03:49 2006 From: venkatbo at yahoo.com (Venkat Bommakanti) Date: Tue, 10 Oct 2006 11:03:49 -0700 (PDT) Subject: [Distutils] compiler selection with bdist_egg of setuptools Message-ID: <20061010180349.78059.qmail@web30509.mail.mud.yahoo.com> >>While compiling some py/C source pkgs, such as: >> . RuleDispatch-0.5a0.dev-r2115.tar.gz >> . cElementTree-1.0.5-20051216.zip >> . PyProtocols-1.0a0dev_r2082.zip >> . Cheetah-2.0rc7.tar.gz >>I noticed in the logs that each (after being unpacked) has >>the following command invoked: >> setup.py -q bdist_egg --dist-dir /tmp/.... >> >>Since selecting a 'compiler' is not an option for the >>"bdist_egg" command, are even the settings in >>~/.pydistutils.cfg file: >> [build] >> compiler = unix >> [bdist_egg] >> plat_name = ppc >>picked up ? >> >>They appear not to be picked up, and I'm suspecting that >>since the [build] settings are valid only for the "build" command, >>it will not have any effect on the "bdist_egg" command used >>by all the above comps. Is that accurate ? >No, it's not accurate. bdist_egg invokes various build commands >(such as build_ext) which get their default --compiler setting from >the [build] configuration. I retried, this time with the same entries in the setup.cfg file of TG: [build] compiler = unix build_temp = [bdist_egg] plat_name = ppc verbose = 1 keep_temp = 1 dist_dir = I can see the verbose logs intially pick up all these settings: ... option dict for 'bdist_egg' command: {'dist_dir': ('setup.cfg', ''), 'keep_temp': ('setup.cfg', '1'), 'plat_name': ('setup.cfg', 'ppc'), 'verbose': ('setup.cfg', '1')} but only "verbose=1" appears to be used. The comps are all invoked as part of TG's: easy_install -f . --prefix=my_usr -s my_usr/bin TurboGears but their respective setup commands get invoked as: setup.py -q bdist_egg --dist-dir /tmp/.... So, my attempts to specify/use from setup.cfg are getting overriden by the above "setup.py -q --dist..." call. This is the snippet that seems to indicate that: ... option dict for 'bdist_egg' command: {'dist_dir': ('command line', '/tmp/easy_install-3XaB58/RuleDispatch-0.5a0.dev-r2115/egg-dist-tmp-wG1geg')} ... >>If plat_name is not picked up, how is setuptools figuring >>out that the platform is i686 (given that the build is being >>done for a i686 linux box). Verbose logs don't show where/how >>that decision is made... >Try running with $DISTUTILS_DEBUG set to "yes". You will get a ton of >output showing what configuration files are being parsed and how the >options for each command are getting set. Look at what's happening when >the second-level setup is run (i.e., the package being asked to >'bdist_egg'); the first-level (easy_install itself) won't make any difference. I did set $DISTUTILS_DEBUG to "yes". Its unclear how/where the fact that its a "i686" platform is picked up frm... Taking the RuleDispatch comp as an example, the first inkling it is going to use "i686" is this log entry: ... installing library code to build/bdist.linux-i686/egg ... Its unclear how this was decided upon... Is it the platform on whicc the python itslef was build or is it the platform of the gcc (in the path) ? Also, what is unclear is how its being determined that the compiler is "gcc" and how its options are being set. Taking RuleDispatch again as an example, I see this: ... building 'dispatch._speedups' extension creating build/temp.linux-i686-2.4 creating build/temp.linux-i686-2.4/src creating build/temp.linux-i686-2.4/src/dispatch gcc -pthread -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC -I//usr/include/python2.4 -c src/dispatch/_speedups.c -o build/temp.linux-i686-2.4/src/dispatch/_speedups.o gcc -pthread -shared build/temp.linux-i686-2.4/src/dispatch/_speedups.o -o build/lib.linux-i686-2.4/dispatch/_speedups.so ... I looked in python, setuptools & RulesDisptach sources, but couldn't find the answers to these questions. If I knew where gcc/and-its-options were being ascertained, config'd, and used, then I could theoretically intervene and have it point to my gcc and use my settings for it... Again, Philip, thanks a bunch for all the time you've spent helping me on this. /venkat From pierre at saiph.com Tue Oct 10 22:14:06 2006 From: pierre at saiph.com (Pierre Imbaud) Date: Tue, 10 Oct 2006 16:14:06 -0400 Subject: [Distutils] setuptools: ALL source tree? Message-ID: Trying to put setuptools to work, from the svn workspace, with no previous experience of distutils or setuptools (previously had a script making a tarball). As often happens with very good tools, very well documented, some very simple, yet vital pieces of information is missing for the newcomer. my source tree: root setup.py package scripts test doc I dont want to distribute test: internal non-regression tools and data. dont want to distribute doc either. I found NO WAY, with setup.py at the top, to get these files out of the tarball. Am I supposed to make ONE tree with all I want to distribute, and all I dont want distributed in some other tree? Like: root dist setup.py package scripts test doc Its no big deal, Im redesigning the layering. But then, the only way to say: "I dont want this distributed any more" is by moving the file? Or did I miss some point? Thanks for answering From pje at telecommunity.com Tue Oct 10 22:33:16 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 10 Oct 2006 16:33:16 -0400 Subject: [Distutils] setuptools: ALL source tree? In-Reply-To: Message-ID: <5.1.1.6.0.20061010162901.02735a80@sparrow.telecommunity.com> At 04:14 PM 10/10/2006 -0400, Pierre Imbaud wrote: >Trying to put setuptools to work, from the svn workspace, with no >previous experience of distutils or setuptools (previously had a script >making a tarball). >As often happens with very good tools, very well documented, some very >simple, yet vital pieces of information is missing for the newcomer. >my source tree: >root > setup.py > package > scripts > test > doc > >I dont want to distribute test: internal non-regression tools and >data. dont want to distribute doc either. >I found NO WAY, with setup.py at the top, to get these files out of >the tarball. See: http://python.org/doc/2.4.1/dist/manifest.html and: http://python.org/doc/2.4.1/dist/sdist-cmd.html#sdist-cmd for the full doc on this. The short version: create a MANIFEST.in file in 'root' with these two lines: prune test prune doc Setuptools will then remove anything under those subdirectories from its source lists. The files will not then be included in source distributions. If you do this, by the way, you should make sure that there is nothing your setup.py needs that is in those directories. Otherwise, source distribution tarballs built with the 'sdist' command will not be buildable or installable. From pierre at saiph.com Tue Oct 10 23:28:06 2006 From: pierre at saiph.com (Pierre Imbaud) Date: Tue, 10 Oct 2006 17:28:06 -0400 Subject: [Distutils] setuptools: ALL source tree? In-Reply-To: <5.1.1.6.0.20061010162901.02735a80@sparrow.telecommunity.com> References: <5.1.1.6.0.20061010162901.02735a80@sparrow.telecommunity.com> Message-ID: <452C1066.4010805@saiph.com> Phillip J. Eby wrote : > At 04:14 PM 10/10/2006 -0400, Pierre Imbaud wrote: >> Trying to put setuptools to work, from the svn workspace, with no >> previous experience of distutils or setuptools (previously had a script >> making a tarball). >> As often happens with very good tools, very well documented, some very >> simple, yet vital pieces of information is missing for the newcomer. >> ... > > See: > > http://python.org/doc/2.4.1/dist/manifest.html > > and: > > http://python.org/doc/2.4.1/dist/sdist-cmd.html#sdist-cmd > > for the full doc on this. > > The short version: create a MANIFEST.in file in 'root' with these two lines: > > prune test > prune doc > > Setuptools will then remove anything under those subdirectories from its > source lists. The files will not then be included in source distributions. Works fine! thanks a lot! From venkatbo at yahoo.com Wed Oct 11 04:45:48 2006 From: venkatbo at yahoo.com (Venkat Bommakanti) Date: Tue, 10 Oct 2006 19:45:48 -0700 (PDT) Subject: [Distutils] gcc compiler options' override Message-ID: <20061011024548.6236.qmail@web30503.mail.mud.yahoo.com> Hi, For a couple comps, like: . RuleDispatch-0.5a0.dev-r2115.tar.gz . cElementTree-1.0.5-20051216.zip, ... I'm trying to build and deploy them as eggs... Using the $CC override, I'm able specify the compiler of choice to compile the C sources in these comps. How do I select the compiler options ( -I, -L, -D etc) and have them picked up by the compiler (--compiler=unix) which in turn could be used by distutils.unixcompiler... Thanks, /venkat From pje at telecommunity.com Wed Oct 11 05:10:05 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 10 Oct 2006 23:10:05 -0400 Subject: [Distutils] gcc compiler options' override In-Reply-To: <20061011024548.6236.qmail@web30503.mail.mud.yahoo.com> Message-ID: <5.1.1.6.0.20061010230945.0274dd28@sparrow.telecommunity.com> At 07:45 PM 10/10/2006 -0700, Venkat Bommakanti wrote: >Hi, > >For a couple comps, like: > . RuleDispatch-0.5a0.dev-r2115.tar.gz > . cElementTree-1.0.5-20051216.zip, ... >I'm trying to build and deploy them as eggs... Using the $CC override, >I'm able specify the compiler of choice to compile the C sources >in these comps. How do I select the compiler options ( -I, -L, -D etc) >and have them picked up by the compiler (--compiler=unix) which in >turn could be used by distutils.unixcompiler... Presumably, by overriding $INCLUDE et al. From pje at telecommunity.com Wed Oct 11 05:18:37 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 10 Oct 2006 23:18:37 -0400 Subject: [Distutils] compiler selection with bdist_egg of setuptools In-Reply-To: <20061010180349.78059.qmail@web30509.mail.mud.yahoo.com> Message-ID: <5.1.1.6.0.20061010231116.039bcd50@sparrow.telecommunity.com> At 11:03 AM 10/10/2006 -0700, Venkat Bommakanti wrote: >I can see the verbose logs intially pick up all these settings: > ... > option dict for 'bdist_egg' command: > {'dist_dir': ('setup.cfg', > ''), > 'keep_temp': ('setup.cfg', '1'), > 'plat_name': ('setup.cfg', 'ppc'), > 'verbose': ('setup.cfg', '1')} >but only "verbose=1" appears to be used. The comps are all invoked as >part of TG's: > easy_install -f . --prefix=my_usr -s my_usr/bin TurboGears >but their respective setup commands get invoked as: > setup.py -q bdist_egg --dist-dir /tmp/.... Right, you need to look at the part of the output that comes *after* this, as that will be the processing for the child setup script, and that's the option processing that counts. >So, my attempts to specify/use from setup.cfg are getting overriden by >the above "setup.py -q --dist..." call. This is the snippet that seems to >indicate that: > ... > option dict for 'bdist_egg' command: > {'dist_dir': ('command line', > >'/tmp/easy_install-3XaB58/RuleDispatch-0.5a0.dev-r2115/egg-dist-tmp-wG1geg')} > ... You need to look for the log information that shows the reading of the configuration files; it would be above this point. You would see lines like: reading /some/dir/.pydistutils.cfg reading /some/path/to/setup.cfg etc. Check and see if the config you think you're using is even being read. >I did set $DISTUTILS_DEBUG to "yes". Its unclear how/where the fact that >its a "i686" platform is picked up frm... Taking the RuleDispatch comp as an >example, the first inkling it is going to use "i686" is this log entry: > ... > installing library code to build/bdist.linux-i686/egg > ... >Its unclear how this was decided upon... Is it the platform on whicc the >python >itslef was build or is it the platform of the gcc (in the path) ? It defaults to the platform the distutils intends to build for. To cross-compile, you would have to explicitly override the platform. Which you're doing, but evidently the config file you think you are using is not being read. >Also, what is unclear is how its being determined that the compiler is "gcc" >and how its options are being set. Taking RuleDispatch again as an example, >I see this: > ... >I looked in python, setuptools & RulesDisptach sources, but couldn't find the >answers to these questions. I think you want to look in distutils.unixccompiler. This isn't a setuptools thing. >If I knew where gcc/and-its-options were being >ascertained, config'd, and used, then I could theoretically intervene and have >it point to my gcc and use my settings for it... > >Again, Philip, thanks a bunch for all the time you've spent helping me on >this. >/venkat No problem, just remember that neither distutils nor setuptools officially provide any support whatsoever for cross-compilation. If you actually succeed in getting something to work, I suggest writing up a HowTo or at least a description of what you ended up doing, so that others can benefit from your trials. From venkatbo at yahoo.com Wed Oct 11 05:26:41 2006 From: venkatbo at yahoo.com (Venkat Bommakanti) Date: Tue, 10 Oct 2006 20:26:41 -0700 (PDT) Subject: [Distutils] compiler selection with bdist_egg of setuptools Message-ID: <20061011032641.10524.qmail@web30509.mail.mud.yahoo.com> >>If I knew where gcc/and-its-options were being >>ascertained, config'd, and used, then I could theoretically intervene and have >>it point to my gcc and use my settings for it... >> >>Again, Philip, thanks a bunch for all the time you've spent helping me on >>this. >>/venkat >No problem, just remember that neither distutils nor setuptools officially >provide any support whatsoever for cross-compilation. If you actually >succeed in getting something to work, I suggest writing up a HowTo or at >least a description of what you ended up doing, so that others can benefit >from your trials. I'm getting closer to this, albeit in baby steps... :-) Will surely write one up - as soon as I have this working. If this doesn't workout, may have to just unpack the distros, use makefiles for each and build the cross .so's that ways... Was hoping to levelrage all the work that has already gone into distutils / setuptools. Thanks again, /venkat From jeff at taupro.com Wed Oct 11 17:01:53 2006 From: jeff at taupro.com (Jeff Rush) Date: Wed, 11 Oct 2006 10:01:53 -0500 Subject: [Distutils] A Call for a Presenter on Python Eggs Message-ID: <452D0761.3020104@taupro.com> Eggs are important to Python and will become more so over the next few years, if understood and embraced by the community. They are a key competitive feature - I've been asked so many times what is the Python equivalent to CPAN and finally we're developing an answer. At PyCon 2007 we need a solid set of talks about eggs, broken down into just using them, how to build and distribute them, and then an advanced talk on some of the more exotic aspects of eggs. At PyCon 2006, essentially the 3rd talk I mention above was given to a standing-room-only audience, and then repeated against an hour later as people in the hall could not get in. But missing were the first two talks - many people were not ready for advanced material but sought an introduction. Who in our community is building eggs on a frequent basis, and is good at explaining how they work? Below are three talk outlines I've sketched out, along with the slides from a talk I tossed together for the Dallas Pythoneers group. They need polish and I'm donating them to the cause. Will someone please step forward and address this critical need? For revision, links to the three wiki outline pages appear on: http://us.pycon.org/TX2007/TalkIdeas --- Python Eggs - Using and Installing Them * What is so cool about eggs? Wow me. * What problem were they intended to solve? * What is their relation to distutils? * So how does distutils work then? * How do they compare to CPAN and RPM approaches? * Sound good, how do I enable eggs on my system? * How do I accomplish common operations? o install an egg o test install an egg, to see if it would work o uninstall an egg o list what eggs are installed at the moment o switch between the versions of an egg * I don't trust eggs - how can I partially install one, examine it and then finish the installation? * How do I find out what eggs exist, and their names? * What about py2exe and droplets? Competing approaches? * Show me a walkthru of the cheeseshop - how to find a package. * How do I figure out the author of an egg I'm having trouble with? --- Python Eggs - Creating and Distributing Them * Where does distutils fit in? * What do I need to add to setup.py to package my software as an egg? * How does it work re path configuration files? * What are the issues with zipped eggs; how can my code access my data? * How do I produce cross-platform eggs? * How do I produce platform-specific eggs? * How do I get my eggs registered on the cheeseshop? * How do I upload my eggs, source and binaries? * How can I volunteer as a packager for non-egg authors? * Some eggs are broken in the cheeseshop - how can I help? (Keep this talk simple and focused on _just_ getting your vanilla egg packaged and uploaded. Defer the advanced egg issues to the third talk). --- Python Eggs - Advanced Packaging * What are entry points? * How does dynamic discovery/plugins work? * I hear Trac uses them - how? * What are extras? They sound nifty! * What is a namespace package? When and how do I use them? * How can I have multiple distribution versions? * I hear eggs can detect files under CVS/subversion control - how does that work? * Also eggs have support for units tests - how? * Where do the egg experts hang out? An Old Talk of mine about Eggs (from which to borrow slides/ideas): * Attach:2006-03-25-AboutPythonEggs.odt * Attach:2006-03-25-AboutPythonEggs.pdf -Jeff Co-Chair PyCon 2007 From pje at telecommunity.com Wed Oct 11 17:35:04 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 11 Oct 2006 11:35:04 -0400 Subject: [Distutils] A Call for a Presenter on Python Eggs In-Reply-To: <452D0761.3020104@taupro.com> Message-ID: <5.1.1.6.0.20061011112731.0273d698@sparrow.telecommunity.com> On all of these items, I can certainly help with providing answers to questions that a speaker may or may not know about currently. I don't know if I will be able to give any talks myself; my PyCon plans (if any) are not finalized yet. I do suggest you ask Kevin Dangoor and Ian Bicking if they are interested; both of them are good at presenting the stuff I dreamed up a year or so before, whereas I tend to present stuff I'm still in the middle of developing. :) I also think you might need to break the first talk into two talks, unless the timeslots for talks have expanded significantly this year. :) At 10:01 AM 10/11/2006 -0500, Jeff Rush wrote: >Python Eggs - Using and Installing Them > > * What is so cool about eggs? Wow me. > * What problem were they intended to solve? > * What is their relation to distutils? > * So how does distutils work then? > * How do they compare to CPAN and RPM approaches? > * Sound good, how do I enable eggs on my system? > * How do I accomplish common operations? > o install an egg > o test install an egg, to see if it would work > o uninstall an egg > o list what eggs are installed at the moment > o switch between the versions of an egg > * I don't trust eggs - how can I partially install one, > examine it and then finish the installation? > * How do I find out what eggs exist, and their names? > * What about py2exe and droplets? Competing approaches? > * Show me a walkthru of the cheeseshop - how to find a package. > * How do I figure out the author of an egg I'm having trouble with? From jeff at taupro.com Wed Oct 11 17:01:53 2006 From: jeff at taupro.com (Jeff Rush) Date: Wed, 11 Oct 2006 10:01:53 -0500 Subject: [Distutils] A Call for a Presenter on Python Eggs Message-ID: <452D0761.3020104@taupro.com> Eggs are important to Python and will become more so over the next few years, if understood and embraced by the community. They are a key competitive feature - I've been asked so many times what is the Python equivalent to CPAN and finally we're developing an answer. At PyCon 2007 we need a solid set of talks about eggs, broken down into just using them, how to build and distribute them, and then an advanced talk on some of the more exotic aspects of eggs. At PyCon 2006, essentially the 3rd talk I mention above was given to a standing-room-only audience, and then repeated against an hour later as people in the hall could not get in. But missing were the first two talks - many people were not ready for advanced material but sought an introduction. Who in our community is building eggs on a frequent basis, and is good at explaining how they work? Below are three talk outlines I've sketched out, along with the slides from a talk I tossed together for the Dallas Pythoneers group. They need polish and I'm donating them to the cause. Will someone please step forward and address this critical need? For revision, links to the three wiki outline pages appear on: http://us.pycon.org/TX2007/TalkIdeas --- Python Eggs - Using and Installing Them * What is so cool about eggs? Wow me. * What problem were they intended to solve? * What is their relation to distutils? * So how does distutils work then? * How do they compare to CPAN and RPM approaches? * Sound good, how do I enable eggs on my system? * How do I accomplish common operations? o install an egg o test install an egg, to see if it would work o uninstall an egg o list what eggs are installed at the moment o switch between the versions of an egg * I don't trust eggs - how can I partially install one, examine it and then finish the installation? * How do I find out what eggs exist, and their names? * What about py2exe and droplets? Competing approaches? * Show me a walkthru of the cheeseshop - how to find a package. * How do I figure out the author of an egg I'm having trouble with? --- Python Eggs - Creating and Distributing Them * Where does distutils fit in? * What do I need to add to setup.py to package my software as an egg? * How does it work re path configuration files? * What are the issues with zipped eggs; how can my code access my data? * How do I produce cross-platform eggs? * How do I produce platform-specific eggs? * How do I get my eggs registered on the cheeseshop? * How do I upload my eggs, source and binaries? * How can I volunteer as a packager for non-egg authors? * Some eggs are broken in the cheeseshop - how can I help? (Keep this talk simple and focused on _just_ getting your vanilla egg packaged and uploaded. Defer the advanced egg issues to the third talk). --- Python Eggs - Advanced Packaging * What are entry points? * How does dynamic discovery/plugins work? * I hear Trac uses them - how? * What are extras? They sound nifty! * What is a namespace package? When and how do I use them? * How can I have multiple distribution versions? * I hear eggs can detect files under CVS/subversion control - how does that work? * Also eggs have support for units tests - how? * Where do the egg experts hang out? An Old Talk of mine about Eggs (from which to borrow slides/ideas): * Attach:2006-03-25-AboutPythonEggs.odt * Attach:2006-03-25-AboutPythonEggs.pdf -Jeff Co-Chair PyCon 2007 -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations.html From bkirsch at osafoundation.org Thu Oct 12 21:02:28 2006 From: bkirsch at osafoundation.org (Brian Kirsch) Date: Thu, 12 Oct 2006 09:02:28 -1000 Subject: [Distutils] Distutils-SIG Digest, Vol 42, Issue 14 In-Reply-To: References: Message-ID: <452E9144.2010501@osafoundation.org> Hi Jeff, I presented last year at PyCon a talk on Internationalizing Chandler. As part of OSAF's effort to provide a clean an easy to use internationalization framework for Python, we developed the EggTranslations API which was completed about two months ago. This API written as an egg provides a framework for other eggs to easily localize. The EggTranslations API is currently leveraged in our product Chandler (www.osafoundation.org) and has proved to be easy to use and extremely effective. I think a standalone talk on EggTranslations would be both a great follow up to last year's Internationalizing Chandler presentation and very much in line with this years PyCon focus on Python eggs. More info on EggTranslations is provided here in the README: http://websvn.osafoundation.org/filedetails.php?repname=chandler&path=%2Ftrunk%2Fchandler%2Fprojects%2FEggTranslations-Plugin%2FREADME&rev=0&sc=0 Here is a link to the source which contains detailed API documentation: http://websvn.osafoundation.org/filedetails.php?repname=chandler&path=%2Ftrunk%2Fchandler%2Fprojects%2FEggTranslations-Plugin%2Fegg_translations.py&rev=0&sc=0 Thanks, Brian -- Brian Kirsch Internationalization Architect / Mail Service Engineer Open Source Applications Foundation 543 Howard Street 5th Floor San Francisco, CA 94105 http://www.osafoundation.org distutils-sig-request at python.org wrote: > Send Distutils-SIG mailing list submissions to > distutils-sig at python.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://mail.python.org/mailman/listinfo/distutils-sig > or, via email, send a message with subject or body 'help' to > distutils-sig-request at python.org > > You can reach the person managing the list at > distutils-sig-owner at python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Distutils-SIG digest..." > > > Today's Topics: > > 1. A Call for a Presenter on Python Eggs (Jeff Rush) > 2. Re: A Call for a Presenter on Python Eggs (Phillip J. Eby) > 3. A Call for a Presenter on Python Eggs (Jeff Rush) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Wed, 11 Oct 2006 10:01:53 -0500 > From: Jeff Rush > Subject: [Distutils] A Call for a Presenter on Python Eggs > To: python-announce-list at python.org, python-list at python.org, > distutils-sig at python.org, dfwpython at dfwpython.org > Message-ID: <452D0761.3020104 at taupro.com> > Content-Type: text/plain; charset=ISO-8859-1 > > Eggs are important to Python and will become more so over the next few years, > if understood and embraced by the community. They are a key competitive > feature - I've been asked so many times what is the Python equivalent to CPAN > and finally we're developing an answer. > > At PyCon 2007 we need a solid set of talks about eggs, broken down into just > using them, how to build and distribute them, and then an advanced talk on > some of the more exotic aspects of eggs. At PyCon 2006, essentially the 3rd > talk I mention above was given to a standing-room-only audience, and then > repeated against an hour later as people in the hall could not get in. But > missing were the first two talks - many people were not ready for advanced > material but sought an introduction. > > Who in our community is building eggs on a frequent basis, and is good at > explaining how they work? > > Below are three talk outlines I've sketched out, along with the slides from a > talk I tossed together for the Dallas Pythoneers group. They need polish and > I'm donating them to the cause. > > Will someone please step forward and address this critical need? > > For revision, links to the three wiki outline pages appear on: > http://us.pycon.org/TX2007/TalkIdeas > > --- > Python Eggs - Using and Installing Them > > * What is so cool about eggs? Wow me. > * What problem were they intended to solve? > * What is their relation to distutils? > * So how does distutils work then? > * How do they compare to CPAN and RPM approaches? > * Sound good, how do I enable eggs on my system? > * How do I accomplish common operations? > o install an egg > o test install an egg, to see if it would work > o uninstall an egg > o list what eggs are installed at the moment > o switch between the versions of an egg > * I don't trust eggs - how can I partially install one, > examine it and then finish the installation? > * How do I find out what eggs exist, and their names? > * What about py2exe and droplets? Competing approaches? > * Show me a walkthru of the cheeseshop - how to find a package. > * How do I figure out the author of an egg I'm having trouble with? > > --- > Python Eggs - Creating and Distributing Them > > * Where does distutils fit in? > * What do I need to add to setup.py to package my software as an egg? > * How does it work re path configuration files? > * What are the issues with zipped eggs; how can my code access my data? > * How do I produce cross-platform eggs? > * How do I produce platform-specific eggs? > * How do I get my eggs registered on the cheeseshop? > * How do I upload my eggs, source and binaries? > * How can I volunteer as a packager for non-egg authors? > * Some eggs are broken in the cheeseshop - how can I help? > > (Keep this talk simple and focused on _just_ getting your vanilla egg packaged > and uploaded. Defer the advanced egg issues to the third talk). > > --- > Python Eggs - Advanced Packaging > > * What are entry points? > * How does dynamic discovery/plugins work? > * I hear Trac uses them - how? > * What are extras? They sound nifty! > * What is a namespace package? When and how do I use them? > * How can I have multiple distribution versions? > * I hear eggs can detect files under CVS/subversion > control - how does that work? > * Also eggs have support for units tests - how? > * Where do the egg experts hang out? > > An Old Talk of mine about Eggs (from which to borrow slides/ideas): > > * Attach:2006-03-25-AboutPythonEggs.odt > * Attach:2006-03-25-AboutPythonEggs.pdf > > -Jeff > Co-Chair PyCon 2007 > > > ------------------------------ > > Message: 2 > Date: Wed, 11 Oct 2006 11:35:04 -0400 > From: "Phillip J. Eby" > Subject: Re: [Distutils] A Call for a Presenter on Python Eggs > To: Jeff Rush , distutils-sig at python.org > Message-ID: > <5.1.1.6.0.20061011112731.0273d698 at sparrow.telecommunity.com> > Content-Type: text/plain; charset="us-ascii"; format=flowed > > On all of these items, I can certainly help with providing answers to > questions that a speaker may or may not know about currently. I don't know > if I will be able to give any talks myself; my PyCon plans (if any) are not > finalized yet. > > I do suggest you ask Kevin Dangoor and Ian Bicking if they are interested; > both of them are good at presenting the stuff I dreamed up a year or so > before, whereas I tend to present stuff I'm still in the middle of > developing. :) > > I also think you might need to break the first talk into two talks, unless > the timeslots for talks have expanded significantly this year. :) > > At 10:01 AM 10/11/2006 -0500, Jeff Rush wrote: > >> Python Eggs - Using and Installing Them >> >> * What is so cool about eggs? Wow me. >> * What problem were they intended to solve? >> * What is their relation to distutils? >> * So how does distutils work then? >> * How do they compare to CPAN and RPM approaches? >> * Sound good, how do I enable eggs on my system? >> * How do I accomplish common operations? >> o install an egg >> o test install an egg, to see if it would work >> o uninstall an egg >> o list what eggs are installed at the moment >> o switch between the versions of an egg >> * I don't trust eggs - how can I partially install one, >> examine it and then finish the installation? >> * How do I find out what eggs exist, and their names? >> * What about py2exe and droplets? Competing approaches? >> * Show me a walkthru of the cheeseshop - how to find a package. >> * How do I figure out the author of an egg I'm having trouble with? >> > > > > ------------------------------ > > Message: 3 > Date: Wed, 11 Oct 2006 10:01:53 -0500 > From: Jeff Rush > Subject: [Distutils] A Call for a Presenter on Python Eggs > To: python-announce-list at python.org, python-list at python.org, > distutils-sig at python.org, dfwpython at dfwpython.org > Message-ID: <452D0761.3020104 at taupro.com> > Content-Type: text/plain; charset="us-ascii" > > Eggs are important to Python and will become more so over the next few years, > if understood and embraced by the community. They are a key competitive > feature - I've been asked so many times what is the Python equivalent to CPAN > and finally we're developing an answer. > > At PyCon 2007 we need a solid set of talks about eggs, broken down into just > using them, how to build and distribute them, and then an advanced talk on > some of the more exotic aspects of eggs. At PyCon 2006, essentially the 3rd > talk I mention above was given to a standing-room-only audience, and then > repeated against an hour later as people in the hall could not get in. But > missing were the first two talks - many people were not ready for advanced > material but sought an introduction. > > Who in our community is building eggs on a frequent basis, and is good at > explaining how they work? > > Below are three talk outlines I've sketched out, along with the slides from a > talk I tossed together for the Dallas Pythoneers group. They need polish and > I'm donating them to the cause. > > Will someone please step forward and address this critical need? > > For revision, links to the three wiki outline pages appear on: > http://us.pycon.org/TX2007/TalkIdeas > > --- > Python Eggs - Using and Installing Them > > * What is so cool about eggs? Wow me. > * What problem were they intended to solve? > * What is their relation to distutils? > * So how does distutils work then? > * How do they compare to CPAN and RPM approaches? > * Sound good, how do I enable eggs on my system? > * How do I accomplish common operations? > o install an egg > o test install an egg, to see if it would work > o uninstall an egg > o list what eggs are installed at the moment > o switch between the versions of an egg > * I don't trust eggs - how can I partially install one, > examine it and then finish the installation? > * How do I find out what eggs exist, and their names? > * What about py2exe and droplets? Competing approaches? > * Show me a walkthru of the cheeseshop - how to find a package. > * How do I figure out the author of an egg I'm having trouble with? > > --- > Python Eggs - Creating and Distributing Them > > * Where does distutils fit in? > * What do I need to add to setup.py to package my software as an egg? > * How does it work re path configuration files? > * What are the issues with zipped eggs; how can my code access my data? > * How do I produce cross-platform eggs? > * How do I produce platform-specific eggs? > * How do I get my eggs registered on the cheeseshop? > * How do I upload my eggs, source and binaries? > * How can I volunteer as a packager for non-egg authors? > * Some eggs are broken in the cheeseshop - how can I help? > > (Keep this talk simple and focused on _just_ getting your vanilla egg packaged > and uploaded. Defer the advanced egg issues to the third talk). > > --- > Python Eggs - Advanced Packaging > > * What are entry points? > * How does dynamic discovery/plugins work? > * I hear Trac uses them - how? > * What are extras? They sound nifty! > * What is a namespace package? When and how do I use them? > * How can I have multiple distribution versions? > * I hear eggs can detect files under CVS/subversion > control - how does that work? > * Also eggs have support for units tests - how? > * Where do the egg experts hang out? > > An Old Talk of mine about Eggs (from which to borrow slides/ideas): > > * Attach:2006-03-25-AboutPythonEggs.odt > * Attach:2006-03-25-AboutPythonEggs.pdf > > -Jeff > Co-Chair PyCon 2007 > From ajones at thegrantinstitute.com Fri Oct 13 05:31:44 2006 From: ajones at thegrantinstitute.com (Anthony Jones) Date: Thu, 12 Oct 2006 20:31:44 -0700 Subject: [Distutils] Professional Grant Proposal Writing Workshop (University of Washington, Seattle - December 2006) Message-ID: <20061012203144.2652859@thegrantinstitute.com> -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20061012/cccb6ef5/attachment.htm From mudengke at gmail.com Fri Oct 13 09:06:04 2006 From: mudengke at gmail.com (=?UTF-8?B?5oWV55m756eR?=) Date: Fri, 13 Oct 2006 15:06:04 +0800 Subject: [Distutils] your setuptools egg seems have a "Trojan" virus Message-ID: hi, I got a virus warning when I Dodnloading "setuptools-0.6c3-py2.5.egg.zip" "TrojanSpy.Delf.s" , can you check the "stuptools" zip file this is the detailed, I download the Dajango 0.95 from their official site DJangoproject.com but install didn't work because the"ez_setup.py" is a old version so i got a new version at"http://peak.telecommunity.com/DevCenter/setuptools" this time it goes well but when it downloading "setuptools-0.6c3-py2.5.egg" I GOT that Trojan virus warning. I think it's impossible ,anyway plese check this url"http://cheeseshop.python.org/packages/2.5/s/setuptools/setuptools-0.6c3-py2.5.egg" From pje at telecommunity.com Fri Oct 13 17:24:50 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 13 Oct 2006 11:24:50 -0400 Subject: [Distutils] your setuptools egg seems have a "Trojan" virus In-Reply-To: Message-ID: <5.1.1.6.0.20061013112112.026e5c90@sparrow.telecommunity.com> At 03:06 PM 10/13/2006 +0800, ? ??????' wrote: >hi, > I got a virus warning when I Dodnloading "setuptools-0.6c3-py2.5.egg.zip" > "TrojanSpy.Delf.s" , > can you check the "stuptools" zip file No, I can't, since I don't know what you mean by "check". I don't know what TronjanSpy.Delf.s *is*. Note that I built the current cheeseshop release of setuptools on a *Linux* machine, so I don't think there's any way it can be infected with a Windows virus. The only possible vector for that would be if the cli.exe and gui.exe files were affected, since I build those on Windows. So if that was what the warning was about, let me know. Otherwise, you should report this as a false positive to your antivirus vendor, so that they can fix their program. It's likely that it is seeing some random data in the zipfile that happens to resemble some portion of the trojan in question. From jjl at pobox.com Sat Oct 14 16:20:10 2006 From: jjl at pobox.com (John J Lee) Date: Sat, 14 Oct 2006 14:20:10 +0000 (UTC) Subject: [Distutils] [kid-discuss] installing without setuptools In-Reply-To: <1160828848.8103.506.camel@www.venix.com> References: <1160773106.8103.464.camel@www.venix.com> <1160828848.8103.506.camel@www.venix.com> Message-ID: Apologies for being unable to resist going over old ground by responding to this (moved here from the kid mailing list). On Sat, 14 Oct 2006, Python wrote: > On Fri, 2006-10-13 at 22:25 +0100, John J Lee wrote: >> On Fri, 13 Oct 2006, Python wrote: >> [...] >>> I've also had grief with easy_install >>> many options did not seem to work >> >> Which ones? > > I can't say now. It seemed to be differences within packages and my > computer environment. I did not keep careful notes. [...] > Phillip suggested > -meb. > which finally worked, but was not obvious from the docs. I see. ISTM that the reason you had trouble is that you had your own ideas about how things should work, rather than following the standard setuptools way of doing things (which it certainly seems weren't arbitrarily chosen, but followed from actual, carefully considered, requirements). It's fair enough to do things your way, but one must always expect to need to expend a bit of effort in trade for "do it *my* way, not the standard way", IMNSHO. You don't get everything for free. OTOH, I don't think I ever managed to get vanilla distutils to install things anywhere but site-packages (admittedly I didn't have a serious need for it, so I gave up after a few attempts), and in contrast I think I managed to do it on my first attempt when using easy_install. So maybe you solved that puzzle with plain-distutils and consider "install it in this directory, I'll be responsible for putting it on sys.path" part of the standard distutils experience... But as I said on the kid list, you're right that easy_install would benefit from more tightly end-user targetted docs (and ISTR PJE hopes to write some later). [slightly re-ordered from original post] > Even the "just download the package and do nothing" option failed. Why is that an "Even"? Downloading the package is bound to fail sometimes given that setuptools tries to support old or uncooperative systems like Sourceforge, arbitrary distutils projects (which can do their own thing with naming conventions), and (ISTR) loosely-defined semantics of standard setup.py parameters like download_url. If you dislike that feature, do not use it. I guess ease of use is subjective, but when I tried it yesterday -- to help out on the kid list -- I found I needed no docs other than easy_install --help to figure out that I could just download stuff myself and run "easy_install --no-deps " (again, sure, "-meb." is NOT obvious -- that's the price of doing stuff your own way, and not a terribly high price as it goes, ISTM). >>> other things simply went wrong Would be nice to know what, if not already reported. [...] >>> The operation is rather opaque and the documentation kind of assumed the >>> simple cases just worked. When operations fail, the trouble shooting >>> steps are not obvious. >> >> it's obvious with plain-distutils? You're a better programmer than I am >> :-) > I doubt it. > > All of the files and configuration info for distutils is in > understandable file formats. eggs are rather difficult if you can't get > the egg manipulation tools to work. [...] But most .eggs, as I understand it, are just .zip files, with a directory inside containing arbitrary metadata files, with some standard metadata for setuptools, which is documented: http://peak.telecommunity.com/DevCenter/EggFormats (only "most" eggs because some are unzipped, and some are implemented with an .egg-info link -- both of which are also documented above) John From parlar at gmail.com Sat Oct 14 17:10:05 2006 From: parlar at gmail.com (Jay Parlar) Date: Sat, 14 Oct 2006 11:10:05 -0400 Subject: [Distutils] numpy build fails in distutils code Message-ID: I've tried asking this on the numpy list (a few times), but unfortunately no response. Since the error *appears* to be inside distutils, I thought maybe someone here would recognize it. Anyway, I'm trying to build the lastest numpy RC with Python 2.5 on OS X 10.3.9, and this is the fun that occurrs: jayparlar$ python setup.py build Running from numpy source directory. F2PY Version 2_3296 blas_opt_info: FOUND: extra_link_args = ['-Wl,-framework', '-Wl,Accelerate'] define_macros = [('NO_ATLAS_INFO', 3)] extra_compile_args = ['-faltivec', '-I/System/Library/Frameworks/vecLib.framework/Headers'] lapack_opt_info: FOUND: extra_link_args = ['-Wl,-framework', '-Wl,Accelerate'] define_macros = [('NO_ATLAS_INFO', 3)] extra_compile_args = ['-faltivec'] running build running config_fc running build_src building py_modules sources creating build creating build/src.macosx-10.3-fat-2.5 creating build/src.macosx-10.3-fat-2.5/numpy creating build/src.macosx-10.3-fat-2.5/numpy/distutils building extension "numpy.core.multiarray" sources creating build/src.macosx-10.3-fat-2.5/numpy/core Generating build/src.macosx-10.3-fat-2.5/numpy/core/config.h customize NAGFCompiler customize AbsoftFCompiler customize IbmFCompiler Could not locate executable g77 Could not locate executable f77 Could not locate executable gfortran Could not locate executable f95 customize GnuFCompiler customize Gnu95FCompiler customize G95FCompiler customize GnuFCompiler customize Gnu95FCompiler customize NAGFCompiler customize NAGFCompiler using config C compiler: gcc -arch ppc -arch i386 -isysroot /Developer/SDKs/MacOSX10.4u.sdk -fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused-madd -fno-common -dynamic -DNDEBUG -g -O3 compile options: '-I/Library/Frameworks/Python.framework/Versions/2.5/include/python2.5 -Inumpy/core/src -Inumpy/core/include -I/Library/Frameworks/Python.framework/Versions/2.5/include/python2.5 -c' gcc: _configtest.c gcc: cannot specify -o with -c or -S and multiple compilations gcc: cannot specify -o with -c or -S and multiple compilations failure. removing: _configtest.c _configtest.o numpy/core/setup.py:50: DeprecationWarning: raising a string exception is deprecated raise "ERROR: Failed to test configuration" Traceback (most recent call last): File "setup.py", line 89, in setup_package() File "setup.py", line 82, in setup_package configuration=configuration ) File "/Users/jayparlar/Desktop/numpy-1.0rc2/numpy/distutils/core.py", line 174, in setup return old_setup(**new_attr) File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/core.py", line 151, in setup dist.run_commands() File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/dist.py", line 974, in run_commands self.run_command(cmd) File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/dist.py", line 994, in run_command cmd_obj.run() File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/command/build.py", line 112, in run self.run_command(cmd_name) File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/cmd.py", line 333, in run_command self.distribution.run_command(command) File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/dist.py", line 994, in run_command cmd_obj.run() File "/Users/jayparlar/Desktop/numpy-1.0rc2/numpy/distutils/command/build_src.py", line 87, in run self.build_sources() File "/Users/jayparlar/Desktop/numpy-1.0rc2/numpy/distutils/command/build_src.py", line 106, in build_sources self.build_extension_sources(ext) File "/Users/jayparlar/Desktop/numpy-1.0rc2/numpy/distutils/command/build_src.py", line 212, in build_extension_sources sources = self.generate_sources(sources, ext) File "/Users/jayparlar/Desktop/numpy-1.0rc2/numpy/distutils/command/build_src.py", line 270, in generate_sources source = func(extension, build_dir) File "numpy/core/setup.py", line 50, in generate_config_h raise "ERROR: Failed to test configuration" ERROR: Failed to test configuration Jay-Computer:~/Desktop/numpy-1.0rc2 jayparlar$ It's failing on this: result = config_cmd.try_run(tc,include_dirs=[python_include], library_dirs = default_lib_dirs) if not result: raise "ERROR: Failed to test configuration" Here, config_cmd is a distutils.command.config.try_run Does anyone recognize this error? I'm looking in the direction of the "gcc: cannot specify -o with -c or -S and multiple compilations" as the root of my problem. Some weird combination of gcc3.3, Python2.5 and OS X 10.3.9 seems to be causing it. I'd really appreciate any insight into this, but totally understand if no one has any ideas :) Jay P. From robert.kern at gmail.com Sat Oct 14 18:27:35 2006 From: robert.kern at gmail.com (Robert Kern) Date: Sat, 14 Oct 2006 12:27:35 -0400 Subject: [Distutils] numpy build fails in distutils code In-Reply-To: References: Message-ID: Jay Parlar wrote: > I've tried asking this on the numpy list (a few times), but > unfortunately no response. Since the error *appears* to be inside > distutils, I thought maybe someone here would recognize it. No, it's not a problem in distutils. Please go back to the numpy list where I will respond to your message. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From pje at telecommunity.com Sat Oct 14 18:31:10 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 14 Oct 2006 12:31:10 -0400 Subject: [Distutils] numpy build fails in distutils code In-Reply-To: Message-ID: <5.1.1.6.0.20061014122954.026e79e8@sparrow.telecommunity.com> At 11:10 AM 10/14/2006 -0400, Jay Parlar wrote: >I've tried asking this on the numpy list (a few times), but >unfortunately no response. Since the error *appears* to be inside >distutils, I thought maybe someone here would recognize it. Nope, it's a numpy.distutils error; notice that the bottom of the traceback is all from numpy code. Distutils also has no "config" command, so that bit's coming from numpy.distutils also. From parlar at gmail.com Sat Oct 14 19:24:11 2006 From: parlar at gmail.com (Jay Parlar) Date: Sat, 14 Oct 2006 13:24:11 -0400 Subject: [Distutils] numpy build fails in distutils code In-Reply-To: <5.1.1.6.0.20061014122954.026e79e8@sparrow.telecommunity.com> References: <5.1.1.6.0.20061014122954.026e79e8@sparrow.telecommunity.com> Message-ID: On 10/14/06, Phillip J. Eby wrote: > At 11:10 AM 10/14/2006 -0400, Jay Parlar wrote: > >I've tried asking this on the numpy list (a few times), but > >unfortunately no response. Since the error *appears* to be inside > >distutils, I thought maybe someone here would recognize it. > > Nope, it's a numpy.distutils error; notice that the bottom of the traceback > is all from numpy code. Distutils also has no "config" command, so that > bit's coming from numpy.distutils also. > > Yeah, the exception is generated in numpy code, but the reason the exception is being raised is because of the 'if not result' check. The 'result' variable gets its value assigned by the config_cmd.try_run() method, which comes from distutils. The relevant numpy code that creates the 'config_cmd' object is: def get_cmd(cmdname, _cache={}): if not _cache.has_key(cmdname): import distutils.core dist = distutils.core._setup_distribution if dist is None: from distutils.errors import DistutilsInternalError raise DistutilsInternalError( 'setup distribution instance not initialized') cmd = dist.get_command_obj(cmdname) _cache[cmdname] = cmd return _cache[cmdname] Could it be that the Python 2.5 universal binary is compiled with a different version of gcc? Would that cause an error? My gcc is 3.3, but if the 2.5 binary was built on OS X Tiger, then the gcc was probably 4.0. Would distutils look at that when trying to build code? Jay P. From venkatbo at yahoo.com Sat Oct 14 22:35:56 2006 From: venkatbo at yahoo.com (Venkat Bommakanti) Date: Sat, 14 Oct 2006 13:35:56 -0700 (PDT) Subject: [Distutils] Question on distutils.sysconfig.get_config_vars( )... Message-ID: <20061014203556.49080.qmail@web30512.mail.mud.yahoo.com> Hi, In: get_config_vars(*args): global _config_vars if _config_vars is None: ... else: ... ... even on the first invocation, it appears _config_vars has some valid data, supposedly picked up from the makefile of the build python... But can't seem to track down where its actually reading/stuffing info into _config_vars. The build system is i686 linux. That said, I don't even see the _init_posix getting invoked for it to read the makefile. In my attemp to cross compile a C extension on a i686 linux box targetting a ppc linux box, I'm hoping to intervene and have it read the config info from the corresponding target python's makefile, and not that of the build python's... I tried asking on comp.lang.python, but to no avail... Thanks much. /venkat From swamidass at gmail.com Sun Oct 15 04:55:15 2006 From: swamidass at gmail.com (S Joshua Swamidass) Date: Sat, 14 Oct 2006 19:55:15 -0700 Subject: [Distutils] setuputils cvs based source detection? Message-ID: Here is the situation: I have a large project in one big cvs tree. i want to be able to release in differen't packages from this tree using different setup.py's. My project is organized like: ./ (root dir) setup_A.py #package = ['A'] setup_B.py #package = ['B'] A/ __init__.py moduleA.py B/ __init__.py moduleB.py The bdist command works fine with this setup. If i use the distutils.core.setup, the sdist command works correctly too. So here is the problem: using setuputils.setup, the sdist command will package the WHOLE source tree (both directories A and B) when using either of the setup_*.py. I'm quite certain this is because both A/ and B/ are under cvs source control, and setup assumes that the whole source tree is required for the sdist. How do i turn this behaivior off or develop a work around? Thanks! S Joshua Swamidass From pje at telecommunity.com Sun Oct 15 05:18:04 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 14 Oct 2006 23:18:04 -0400 Subject: [Distutils] setuputils cvs based source detection? In-Reply-To: Message-ID: <5.1.1.6.0.20061014231130.026e6008@sparrow.telecommunity.com> At 07:55 PM 10/14/2006 -0700, S Joshua Swamidass wrote: >Here is the situation: > >I have a large project in one big cvs tree. i want to be able to >release in differen't packages from this tree using different >setup.py's. My project is organized like: > >./ (root dir) >setup_A.py #package = ['A'] >setup_B.py #package = ['B'] >A/ > __init__.py > moduleA.py >B/ > __init__.py > moduleB.py > >The bdist command works fine with this setup. If i use the >distutils.core.setup, the sdist command works correctly too. So here >is the problem: using setuputils.setup, the sdist command will package >the WHOLE source tree (both directories A and B) when using either of >the setup_*.py. > >I'm quite certain this is because both A/ and B/ are under cvs source >control, and setup assumes that the whole source tree is required for >the sdist. How do i turn this behaivior off or develop a work around? setuptools doesn't provide a way to support this behavior. Even if you were to manually modify MANIFEST.in to exclude the files you didn't want, you'd have to change it every time you issued a release, and your source distributions wouldn't be distutils-standard or easy_install-compatible anyway, because you wouldn't have a *single* "setup.py" file in the distribution, and having a "setup_A.py" isn't the same thing. My suggestion would be for you to change how you're doing this, so that you have a single setup.py for each thing you want to distribute separately, or else distribute the whole thing as one giant package. If you want to minimize the amount of directory reorganization needed, one possible approach is to put a setup.py in each of A and B, and adjust the package_dir setting of setup() accordingly. There are some drawbacks to that layout, but at least you would be able to keep your package directories where they are now. From swamidass at gmail.com Sun Oct 15 05:30:37 2006 From: swamidass at gmail.com (S Joshua Swamidass) Date: Sat, 14 Oct 2006 20:30:37 -0700 Subject: [Distutils] setuputils cvs based source detection? In-Reply-To: <5.1.1.6.0.20061014231130.026e6008@sparrow.telecommunity.com> References: <5.1.1.6.0.20061014231130.026e6008@sparrow.telecommunity.com> Message-ID: Hmm, As much as I like setuputils, this might be enough reason for me to stay with distutils which seems to handle this setup gracefully. The actually setup i have is significantly more complex, so creating seperate cvs repositories for each would not necessarily be possilbe and most definetely not easy. Thanks for the help any way, Josh On 10/14/06, Phillip J. Eby wrote: > At 07:55 PM 10/14/2006 -0700, S Joshua Swamidass wrote: > >Here is the situation: > > > >I have a large project in one big cvs tree. i want to be able to > >release in differen't packages from this tree using different > >setup.py's. My project is organized like: > > > >./ (root dir) > >setup_A.py #package = ['A'] > >setup_B.py #package = ['B'] > >A/ > > __init__.py > > moduleA.py > >B/ > > __init__.py > > moduleB.py > > > >The bdist command works fine with this setup. If i use the > >distutils.core.setup, the sdist command works correctly too. So here > >is the problem: using setuputils.setup, the sdist command will package > >the WHOLE source tree (both directories A and B) when using either of > >the setup_*.py. > > > >I'm quite certain this is because both A/ and B/ are under cvs source > >control, and setup assumes that the whole source tree is required for > >the sdist. How do i turn this behaivior off or develop a work around? > > setuptools doesn't provide a way to support this behavior. > > Even if you were to manually modify MANIFEST.in to exclude the files you > didn't want, you'd have to change it every time you issued a release, and > your source distributions wouldn't be distutils-standard or > easy_install-compatible anyway, because you wouldn't have a *single* > "setup.py" file in the distribution, and having a "setup_A.py" isn't the > same thing. > > My suggestion would be for you to change how you're doing this, so that you > have a single setup.py for each thing you want to distribute separately, or > else distribute the whole thing as one giant package. > > If you want to minimize the amount of directory reorganization needed, one > possible approach is to put a setup.py in each of A and B, and adjust the > package_dir setting of setup() accordingly. There are some drawbacks to > that layout, but at least you would be able to keep your package > directories where they are now. > > From ronaldoussoren at mac.com Sun Oct 15 18:33:39 2006 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Sun, 15 Oct 2006 18:33:39 +0200 Subject: [Distutils] numpy build fails in distutils code In-Reply-To: References: Message-ID: <66CB0135-90DD-4DF6-AACD-1410912166B6@mac.com> On Oct 14, 2006, at 6:27 PM, Robert Kern wrote: > Jay Parlar wrote: >> I've tried asking this on the numpy list (a few times), but >> unfortunately no response. Since the error *appears* to be inside >> distutils, I thought maybe someone here would recognize it. > > No, it's not a problem in distutils. Please go back to the numpy > list where I > will respond to your message. Actually it is a problem in disutils, one that is AFAIK fixed in 2.4.4c1 (and the same patch is in the 2.5 branch and on the trunk). What happens is that the binary release of Python for OSX is built as a universal binary and uses compiler flags that the compiler toolchain on OSX 10.3.9 doesn't understand. Distutils tries to remove those flags on 10.3.9, but I forgot a number of settings that need to be cleaned up. Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3562 bytes Desc: not available Url : http://mail.python.org/pipermail/distutils-sig/attachments/20061015/c653b220/attachment.bin From ronaldoussoren at mac.com Sun Oct 15 18:38:48 2006 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Sun, 15 Oct 2006 18:38:48 +0200 Subject: [Distutils] numpy build fails in distutils code In-Reply-To: <5.1.1.6.0.20061014122954.026e79e8@sparrow.telecommunity.com> References: <5.1.1.6.0.20061014122954.026e79e8@sparrow.telecommunity.com> Message-ID: <071A747B-C764-4929-963D-C0066C07B23F@mac.com> On Oct 14, 2006, at 6:31 PM, Phillip J. Eby wrote: > At 11:10 AM 10/14/2006 -0400, Jay Parlar wrote: >> I've tried asking this on the numpy list (a few times), but >> unfortunately no response. Since the error *appears* to be inside >> distutils, I thought maybe someone here would recognize it. > > Nope, it's a numpy.distutils error; notice that the bottom of the > traceback > is all from numpy code. Distutils also has no "config" command, so > that > bit's coming from numpy.distutils also. Well actually, distutils does have a config command, it doesn't do anything but it is there for autoconf-like feature checks and is meant to be subclassed. At least, that is my understanding from reading the source code. Ronald > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3562 bytes Desc: not available Url : http://mail.python.org/pipermail/distutils-sig/attachments/20061015/cf7ee2e0/attachment.bin From jeremy.kloth at 4suite.org Sun Oct 15 19:31:14 2006 From: jeremy.kloth at 4suite.org (Jeremy Kloth) Date: Sun, 15 Oct 2006 11:31:14 -0600 Subject: [Distutils] setuputils cvs based source detection? In-Reply-To: References: Message-ID: <200610151131.14780.jeremy.kloth@4suite.org> On Saturday 14 October 2006 8:55 pm, S Joshua Swamidass wrote: > Here is the situation: > > I have a large project in one big cvs tree. i want to be able to > release in differen't packages from this tree using different > setup.py's. My project is organized like: > > ./ (root dir) > setup_A.py #package = ['A'] > setup_B.py #package = ['B'] > A/ > __init__.py > moduleA.py > B/ > __init__.py > moduleB.py Check out 4Suite's distutils extensions. They support this kind of layout as that is how 4Suite itself is layed out. It is run with a "master" setup.py file that defers to "package files" for the contents of sub-packages. from Ft.Lib.DistExt import setup setup(name="LargeProject", package_files=['A.pkg', 'B.pkg'], # Other values that are consistent between sub-packages # like any of the metadata fields (including the fields # from PEP 345 like `requires`, `provides`, `obsoletes` # and `requires_python`. ) # This file contains the setup() keywords specific to 'A'. # Either `package` or `name` is required. `package` means the sub-package # name will be, in this case, "LargeProject-A". package = 'A' # or # name = "SomeProjectA" # This file contains the setup() keywords specific to 'B'. package = 'B' Other keywords that are usable as setup() keyword arguments are allowed. This file is treated as a "regular" Python script, so imports and such are allowed. The various classes used for representing data structures are automatically provided in the namespace for the script when it is evaluated for keyword arguments. To work on a single package, call setup.py with a `--package=` (-p) argument. Otherwise it will iterate over all sub-packages sorted by the provides/requires order of inter-package dependencies. The DistExt feature list includes: * .egg distributions * InnoSetup installers for Windows * scripts are real executables on Windows * automatic API documentation generation * a `config` command for saving configuration between multiple runs * gettext (l10n) message catalogs * a "smart" sdist that packages files referenced the all the commands (include the C headers for extension modules) DistExt has been around a long time, since distutils first release in fact, and is in constant use, but fairly under-documented (as it has been 4Suite-only, but I'm looking to change that). I'd suggest looking at 4Suite's setup.py and related package files for a start. -- Jeremy Kloth http://4suite.org/ From parlar at gmail.com Mon Oct 16 15:10:32 2006 From: parlar at gmail.com (Jay Parlar) Date: Mon, 16 Oct 2006 09:10:32 -0400 Subject: [Distutils] numpy build fails in distutils code Message-ID: > On Oct 14, 2006, at 6:27 PM, Robert Kern wrote: > Actually it is a problem in disutils, one that is AFAIK fixed in > 2.4.4c1 (and the same patch is in the 2.5 branch and on the trunk). > > What happens is that the binary release of Python for OSX is built as > a universal binary and uses compiler flags that the compiler > toolchain on OSX 10.3.9 doesn't understand. Distutils tries to remove > those flags on 10.3.9, but I forgot a number of settings that need to > be cleaned up. Perfect, that's what I suspected was happening. I'll just wait for 2.5.1 then. Thanks a lot! Jay P. From ronaldoussoren at mac.com Mon Oct 16 15:23:40 2006 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Mon, 16 Oct 2006 15:23:40 +0200 Subject: [Distutils] numpy build fails in distutils code In-Reply-To: References: Message-ID: <9689293.1161005020896.JavaMail.ronaldoussoren@mac.com> On Monday, October 16, 2006, at 03:11PM, Jay Parlar wrote: >> On Oct 14, 2006, at 6:27 PM, Robert Kern wrote: > >> Actually it is a problem in disutils, one that is AFAIK fixed in >> 2.4.4c1 (and the same patch is in the 2.5 branch and on the trunk). >> >> What happens is that the binary release of Python for OSX is built as >> a universal binary and uses compiler flags that the compiler >> toolchain on OSX 10.3.9 doesn't understand. Distutils tries to remove >> those flags on 10.3.9, but I forgot a number of settings that need to >> be cleaned up. > >Perfect, that's what I suspected was happening. I'll just wait for >2.5.1 then. Thanks a lot! You can also edit /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/config/Makefile. Remove '-arch ppc -arch i386 -isysroot /Developer/SDKs/MacOSX.10.4u.sdk' from both the BASECFLAGS and LDFLAGS. Ronald P.S. I typed those arguments, the actual arguments might be slightly different. > >Jay P. >_______________________________________________ >Distutils-SIG maillist - Distutils-SIG at python.org >http://mail.python.org/mailman/listinfo/distutils-sig > > From pierre at saiph.com Mon Oct 16 22:57:56 2006 From: pierre at saiph.com (Pierre Imbaud) Date: Mon, 16 Oct 2006 16:57:56 -0400 Subject: [Distutils] disappointed by setupTools Message-ID: Im a little bit disappointed by setupTools. Not that much the tool itself, than the learning curve. Maybe the problem at hand is far from simple, and thats what makes the solution so intricate. http://peak.telecommunity.com/DevCenter/setuptools: this document is 100 pages long! (by page I mean: full screen). easyInstall is 40 pages. I spent about 3 days to figure out what goes where. And I realize Im not done. Yet the documentation is remarkable, great presentation, text is precise and clear, when U read it. Makes U feel guilty U dont find what U lookin for, but U dont find it. I just discovered scripts dont get installed directly (in /usr/local/bin), instead a wrapper in installed, that calls the script. So an extra file is involved, and extra code, every time the script is called. Makes some overhead, and it breaks a simple mechanism I use: my script is able to perform many actions, based on its name (much like cp, mv and ln are the same executable). at install, I need to make symlinks to the script, one per action, and this was automated: this is broken by the wrapping. No big deal, just makes things less simple. Good tools have 2 qualities: - When U need simple things, its easy to use. - When U need more sophisticated things, U dont need to give up the tool for a more powerful one: just have to use extra features. Python is a wonderful illustration of this. While setupTools obviously provides the latter, I consider it fails on the former. I decided to use setupTools after reading, and failing to understand, distutils. setupTools seemed both more powerful and simpler. Probably this choice will save me some trouble later. Im not sure it helped so far. From matt at matt-good.net Mon Oct 16 23:46:26 2006 From: matt at matt-good.net (Matt Good) Date: Mon, 16 Oct 2006 17:46:26 -0400 Subject: [Distutils] disappointed by setupTools In-Reply-To: References: Message-ID: <1161035186.5853.7.camel@nny> On Mon, 2006-10-16 at 16:57 -0400, Pierre Imbaud wrote: > I just discovered scripts dont get installed directly (in > /usr/local/bin), instead a wrapper in installed, that calls the > script. So an extra file is involved, and extra code, every time the > script is called. Makes some overhead, and it breaks a simple > mechanism I use: my script is able to perform many actions, based on > its name (much like cp, mv and ln are the same executable). at > install, I need to make symlinks to the script, one per action, and > this was automated: this is broken by the wrapping. No big deal, just > makes things less simple. Actually installing scripts with setuptools is very easy: http://peak.telecommunity.com/DevCenter/setuptools#automatic-script-creation In your entry points simply put the names of the scripts you want and the Python function that script will run. Now you don't need to do your own symlink hack to get it all to work. Just put the functions into a Python module and list them in setup.py. The setuptools method also fixes problems with making scripts work in a cross-platform manner, makes sure your scripts work correctly if someone installs two versions of the same package on their system, and ensures that scripts are run with the correct Python version in case it's not the default Python executable on the system. -- Matt Good From bob at redivi.com Mon Oct 16 23:54:25 2006 From: bob at redivi.com (Bob Ippolito) Date: Mon, 16 Oct 2006 14:54:25 -0700 Subject: [Distutils] disappointed by setupTools In-Reply-To: References: Message-ID: <6a36e7290610161454pf27a691id215e13a7b135345@mail.gmail.com> On 10/16/06, Pierre Imbaud wrote: > Im a little bit disappointed by setupTools. Not that much the tool itself, > than the learning curve. > Maybe the problem at hand is far from simple, and thats what makes the > solution so intricate. > http://peak.telecommunity.com/DevCenter/setuptools: this document is > 100 pages long! (by page I mean: full screen). > easyInstall is 40 pages. > I spent about 3 days to figure out what goes where. And I realize Im > not done. > Yet the documentation is remarkable, great presentation, text is > precise and clear, when U read it. Makes U feel guilty U dont find > what U lookin for, but U dont find it. The current documentation is a reference, not a tutorial. It's long because it's exhaustive. What's needed is different documentation for new users. > I just discovered scripts dont get installed directly (in > /usr/local/bin), instead a wrapper in installed, that calls the > script. So an extra file is involved, and extra code, every time the > script is called. Makes some overhead, and it breaks a simple > mechanism I use: my script is able to perform many actions, based on > its name (much like cp, mv and ln are the same executable). at > install, I need to make symlinks to the script, one per action, and > this was automated: this is broken by the wrapping. No big deal, just > makes things less simple. > Good tools have 2 qualities: > - When U need simple things, its easy to use. > - When U need more sophisticated things, U dont need to give up the > tool for a more powerful one: just have to use extra features. > Python is a wonderful illustration of this. > While setupTools obviously provides the latter, I consider it fails on > the former. > I decided to use setupTools after reading, and failing to understand, > distutils. setupTools seemed both more powerful and simpler. > Probably this choice will save me some trouble later. Im not sure it > helped so far. It doesn't work because you're doing it the wrong way. If you want to see what a command was invoked as, use sys.argv[0]... that's what it's for. Also, you don't *have* to use entry points to make scripts (though it's generally better to)... the scripts kwarg to setup works the same way that it does for distutils. -bob From exarkun at divmod.com Tue Oct 17 03:19:07 2006 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Mon, 16 Oct 2006 21:19:07 -0400 Subject: [Distutils] Setuptools with custom path configuration In-Reply-To: 0 Message-ID: <20061017011907.26151.738841110.divmod.quotient.4392@ohm> I revisited setuptools again tonight hoping to learn a bit more about it. Unfortunately I'm still stuck where I was last time I looked. I'm using Combinator to manage my Python libraries. This means that I have a fair amount of software installed at ~/.local/lib/pythonX.Y/site-packages, a PYTHONPATH which includes only ~/Projects/Divmod/trunk/Combinator, at which location resides a sitecustomize.py which sets up the rest of my path. >From the documentation, and various experimentation, it seems like this configuration is still not supported. What needs to happen for to change this? Jean-Paul From pje at telecommunity.com Tue Oct 17 05:11:05 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 16 Oct 2006 23:11:05 -0400 Subject: [Distutils] Setuptools with custom path configuration In-Reply-To: <20061017011907.26151.738841110.divmod.quotient.4392@ohm> References: <0> Message-ID: <5.1.1.6.0.20061016230741.0270b400@sparrow.telecommunity.com> At 09:19 PM 10/16/2006 -0400, Jean-Paul Calderone wrote: >I revisited setuptools again tonight hoping to learn a bit more about >it. Unfortunately I'm still stuck where I was last time I looked. > >I'm using Combinator to manage my Python libraries. This means that I have >a fair amount of software installed at >~/.local/lib/pythonX.Y/site-packages, a PYTHONPATH which includes only >~/Projects/Divmod/trunk/Combinator, at which location resides a >sitecustomize.py which sets up the rest of my path. > > From the documentation, and various experimentation, it seems like this > configuration is still not supported. It is supported. What problem are you having? As long as you are having setuptools install packages to a directory on PYTHONPATH, you should be fine. If you are installing to a directory that is not on PYTHONPATH, then it has to be a "site" directory, meaning that you add it to sys.path using site.addsitedir(). If you don't do this, then .pth files in the directory will not be processed, and setuptools will not allow installing packages there except in --multi-version mode (which needs an explicit require() operation to add the package to sys.path at runtime, rather than always including it on sys.path at startup). From exarkun at divmod.com Tue Oct 17 14:09:43 2006 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Tue, 17 Oct 2006 08:09:43 -0400 Subject: [Distutils] Setuptools with custom path configuration In-Reply-To: <5.1.1.6.0.20061016230741.0270b400@sparrow.telecommunity.com> Message-ID: <20061017120943.26151.1691049668.divmod.quotient.4814@ohm> On Mon, 16 Oct 2006 23:11:05 -0400, "Phillip J. Eby" wrote: >At 09:19 PM 10/16/2006 -0400, Jean-Paul Calderone wrote: >>I revisited setuptools again tonight hoping to learn a bit more about it. >>Unfortunately I'm still stuck where I was last time I looked. >> >>I'm using Combinator to manage my Python libraries. This means that I have >>a fair amount of software installed at ~/.local/lib/pythonX.Y/site- >>packages, a PYTHONPATH which includes only >>~/Projects/Divmod/trunk/Combinator, at which location resides a >>sitecustomize.py which sets up the rest of my path. >> >> From the documentation, and various experimentation, it seems like this >>configuration is still not supported. > >It is supported. What problem are you having? As long as you are having >setuptools install packages to a directory on PYTHONPATH, you should be >fine. When I run ez_setup.py, it tells me: TEST FAILED: /home/exarkun/.local/lib/python2.4/site-packages/ does NOT support .pth files error: bad install directory or PYTHONPATH ... /home/exarkun/.local/lib/python2.4/site-packages/ is not in PYTHONPATH, but it has been passed to site.addsitedir(). Jean-Paul From pje at telecommunity.com Tue Oct 17 17:32:13 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 17 Oct 2006 11:32:13 -0400 Subject: [Distutils] Setuptools with custom path configuration In-Reply-To: <20061017120943.26151.1691049668.divmod.quotient.4814@ohm> References: <5.1.1.6.0.20061016230741.0270b400@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20061017112819.026b8ff8@sparrow.telecommunity.com> At 08:09 AM 10/17/2006 -0400, Jean-Paul Calderone wrote: >When I run ez_setup.py, it tells me: > >TEST FAILED: /home/exarkun/.local/lib/python2.4/site-packages/ does NOT >support .pth files >error: bad install directory or PYTHONPATH > >... > >/home/exarkun/.local/lib/python2.4/site-packages/ is not in PYTHONPATH, >but it has been passed to site.addsitedir(). But the file that does it is in PYTHONPATH... which means it's not getting run during the test, which uses python -E. Argh. I've never seen a configuration that used (and needed) PYTHONPATH *and* sitecustomize *and* a site directory before. You're right, this configuration isn't really supported well. easy_install expects non-PYTHONPATH site dirs to be supported by Python itself, not by a PYTHONPATH dir causing the other one to be loaded. I'll have to see about a fix. From exarkun at divmod.com Tue Oct 17 17:43:07 2006 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Tue, 17 Oct 2006 11:43:07 -0400 Subject: [Distutils] Setuptools with custom path configuration In-Reply-To: <5.1.1.6.0.20061017112819.026b8ff8@sparrow.telecommunity.com> Message-ID: <20061017154307.26151.509673066.divmod.quotient.4957@ohm> On Tue, 17 Oct 2006 11:32:13 -0400, "Phillip J. Eby" wrote: > [snip] > >You're right, this configuration isn't really supported well. easy_install >expects non-PYTHONPATH site dirs to be supported by Python itself, not by a >PYTHONPATH dir causing the other one to be loaded. I'll have to see about a >fix. > Okay, thanks. Let me know if there's anything I can do to help; testing, etc. Jean-Paul From pje at telecommunity.com Tue Oct 17 18:00:39 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 17 Oct 2006 12:00:39 -0400 Subject: [Distutils] Setuptools with custom path configuration In-Reply-To: <20061017154307.26151.509673066.divmod.quotient.4957@ohm> References: <5.1.1.6.0.20061017112819.026b8ff8@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20061017120021.03bd53d0@sparrow.telecommunity.com> At 11:43 AM 10/17/2006 -0400, Jean-Paul Calderone wrote: >On Tue, 17 Oct 2006 11:32:13 -0400, "Phillip J. Eby" > wrote: >>[snip] >> >>You're right, this configuration isn't really supported >>well. easy_install expects non-PYTHONPATH site dirs to be supported by >>Python itself, not by a PYTHONPATH dir causing the other one to be >>loaded. I'll have to see about a fix. > >Okay, thanks. Let me know if there's anything I can do to help; testing, etc. I'm about to be away from 'net access for a week; will let you know when I get back. From jkgp at tollgatedrive.fsnet.co.uk Tue Oct 17 22:49:55 2006 From: jkgp at tollgatedrive.fsnet.co.uk (Evelina Higgins) Date: Tue, 17 Oct 2006 16:49:55 -0400 Subject: [Distutils] intensively Message-ID: <453541F3.6090007@soundenvision.com> An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20061017/8b0e40f5/attachment-0001.html -------------- next part -------------- A non-text attachment was scrubbed... Name: repaid.gif Type: image/gif Size: 12336 bytes Desc: not available Url : http://mail.python.org/pipermail/distutils-sig/attachments/20061017/8b0e40f5/attachment-0001.gif From S.Pascoe at rl.ac.uk Wed Oct 18 16:55:00 2006 From: S.Pascoe at rl.ac.uk (Pascoe, S (Stephen)) Date: Wed, 18 Oct 2006 15:55:00 +0100 Subject: [Distutils] Distributing executable binaries in eggs Message-ID: I am trying create eggs for a package that relies on a compiled executable that is distributed with the package. I can get the executable into the egg fine. It is declared as package_data and I set zip_safe=False to force easy_install to unpack the egg. My problem is that the executable is always extracted without execution permissions (our platform is UNIX). After some experimenting this appears to be a general problem with the zip file module not honouring execution permissions. I know permissions can be set in zip files because the InfoZip tools do it. Currently I have a hack working that tries to change the permissions on the executable when the package is imported but this won't work if the egg is installed by a different user to the importer. I am wondering if there is a way round this with setuptools. Some sort of hook to fix permissions when unpacking a non-zipsafe egg would be good. Thanks, Stephen. --- Stephen Pascoe 01235 445980 British Atmospheric Data Centre Rutherford Appleton Laboratory, CCLRC -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20061018/23559f07/attachment.htm From ianb at colorstudy.com Thu Oct 19 00:16:42 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Wed, 18 Oct 2006 17:16:42 -0500 Subject: [Distutils] setuptools: but with svn-tag-revision and uncommitted project In-Reply-To: <4523FCAE.8000208@colorstudy.com> References: <4523FCAE.8000208@colorstudy.com> Message-ID: <4536A7CA.8040201@colorstudy.com> Anything on this? Ian Bicking wrote: > I get this exception when I run "python setup.py egg_info" with > --tag-svn-revision and when the project has never been committed (and > hence there's no committed-rev). I think it should instead give 0, or > at least a good error message. (I don't see any problem with 0) > > Traceback (most recent call last): > File "setup.py", line 24, in ? > entry_points=""" > File "/usr/lib/python2.4/distutils/core.py", line 149, in setup > dist.run_commands() > File "/usr/lib/python2.4/distutils/dist.py", line 946, in run_commands > self.run_command(cmd) > File "/usr/lib/python2.4/distutils/dist.py", line 965, in run_command > cmd_obj.ensure_finalized() > File "/usr/lib/python2.4/distutils/cmd.py", line 117, in ensure_finalized > self.finalize_options() > File > "/usr/local/lib/python2.4/site-packages/setuptools-0.6c3-py2.4.egg/setuptools/command/egg_info.py", > line 85, in finalize_options > self.vtags = self.tags() > File > "/usr/local/lib/python2.4/site-packages/setuptools-0.6c3-py2.4.egg/setuptools/command/egg_info.py", > line 179, in tags > ): version += '-r%s' % self.get_svn_revision() > File > "/usr/local/lib/python2.4/site-packages/setuptools-0.6c3-py2.4.egg/setuptools/command/egg_info.py", > line 227, in get_svn_revision > localrev = max([int(m.group(1)) for m in revre.finditer(data)]) > ValueError: max() arg is an empty sequence > From ianb at colorstudy.com Thu Oct 19 00:19:03 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Wed, 18 Oct 2006 17:19:03 -0500 Subject: [Distutils] easy_install --always-copy and setup.py develop In-Reply-To: <452B0738.7060006@colorstudy.com> References: <452B0738.7060006@colorstudy.com> Message-ID: <4536A857.90700@colorstudy.com> Anything on this one too? Ian Bicking wrote: > I have a package (PasteDeploy) installed with "python setup.py develop", > at version 0.9.7dev-r5544. There's a released version 0.9.6. When I do > "easy_install --always-copy --install-dir /some/dir > SomePackageThatRequiresPasteDeploy" it gives me errors like error: > > Could not find suitable distribution for > Requirement.parse('PasteDeploy==0.9.7dev-r5544') (--always-copy skips > system and development eggs) > > There is no requirement like that, it seems to be coming up with that > version specifier on its own. If I try to install PasteDeploy > specifically it works fine. If I remove that egg-link from develop it > works fine. > > (Note: my end goal is to create a bundle of all the eggs that an app > needs to run) From sb at csse.unimelb.edu.au Thu Oct 19 03:11:00 2006 From: sb at csse.unimelb.edu.au (Steven Bird) Date: Thu, 19 Oct 2006 11:11:00 +1000 Subject: [Distutils] Need help switching to setuptools Message-ID: <97e4e62e0610181811s33e5a749ka0b942c3be179b83@mail.gmail.com> Hi, I'm leading the development of NLTK, the Natural Language Toolkit, a suite of Python modules for analyzing text [http://nltk.sourceforge.net/]. I'm strugging to work out how to switch from distutils to setuptools, and am wondering if there's someone out there who can work with me to do the migration. There are several components to the distribution: * a dozen pure python packages * a dependency on numpy * a dependency on elementtree for Python < 2.5 * textbook pdfs to be installed centrally * 200Mb of data (uncompressed) to be installed centrally * similar amount of data to be generated at install time and stored centrally ("models") * directory tree to be stored to user filespace (code samples from the textbook) * locations of all data to be discovered at run-time by the toolkit * easy installation on linux, mac and windows I believe setuptools can do all these things, but I'm stalled on my way up a steep learning curve! The NLTK subversion repository is available for anonymous access here: svn co https://svn.sourceforge.net/svnroot/nltk/trunk/nltk Any help would be gratefully appreciated. Thanks, -Steven Bird From pje at telecommunity.com Thu Oct 19 06:17:15 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 18 Oct 2006 21:17:15 -0700 Subject: [Distutils] setuptools: but with svn-tag-revision and uncommitted project In-Reply-To: <4536A7CA.8040201@colorstudy.com> References: <4523FCAE.8000208@colorstudy.com> <4536A7CA.8040201@colorstudy.com> Message-ID: <7.0.1.0.0.20061018211643.021c1cc0@telecommunity.com> I thought this was already fixed in SVN. Have you updated? (I'm currently on limited connectivity until next week.) At 03:16 PM 10/18/2006, Ian Bicking wrote: >Anything on this? > >Ian Bicking wrote: > > I get this exception when I run "python setup.py egg_info" with > > --tag-svn-revision and when the project has never been committed (and > > hence there's no committed-rev). I think it should instead give 0, or > > at least a good error message. (I don't see any problem with 0) > > > > Traceback (most recent call last): > > File "setup.py", line 24, in ? > > entry_points=""" > > File "/usr/lib/python2.4/distutils/core.py", line 149, in setup > > dist.run_commands() > > File "/usr/lib/python2.4/distutils/dist.py", line 946, in run_commands > > self.run_command(cmd) > > File "/usr/lib/python2.4/distutils/dist.py", line 965, in run_command > > cmd_obj.ensure_finalized() > > File "/usr/lib/python2.4/distutils/cmd.py", line 117, in > ensure_finalized > > self.finalize_options() > > File > > > "/usr/local/lib/python2.4/site-packages/setuptools-0.6c3-py2.4.egg/setuptools/command/egg_info.py", > > > line 85, in finalize_options > > self.vtags = self.tags() > > File > > > "/usr/local/lib/python2.4/site-packages/setuptools-0.6c3-py2.4.egg/setuptools/command/egg_info.py", > > > line 179, in tags > > ): version += '-r%s' % self.get_svn_revision() > > File > > > "/usr/local/lib/python2.4/site-packages/setuptools-0.6c3-py2.4.egg/setuptools/command/egg_info.py", > > > line 227, in get_svn_revision > > localrev = max([int(m.group(1)) for m in revre.finditer(data)]) > > ValueError: max() arg is an empty sequence > > > >_______________________________________________ >Distutils-SIG maillist - Distutils-SIG at python.org >http://mail.python.org/mailman/listinfo/distutils-sig From pje at telecommunity.com Thu Oct 19 06:16:17 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 18 Oct 2006 21:16:17 -0700 Subject: [Distutils] Need help switching to setuptools In-Reply-To: <97e4e62e0610181811s33e5a749ka0b942c3be179b83@mail.gmail.co m> References: <97e4e62e0610181811s33e5a749ka0b942c3be179b83@mail.gmail.com> Message-ID: <7.0.1.0.0.20061018211237.021cf078@telecommunity.com> At 06:11 PM 10/18/2006, Steven Bird wrote: >Hi, > >I'm leading the development of NLTK, the Natural Language Toolkit, a >suite of Python modules for analyzing text >[http://nltk.sourceforge.net/]. I'm strugging to work out how to >switch from distutils to setuptools, and am wondering if there's >someone out there who can work with me to do the migration. > >There are several components to the distribution: >* a dozen pure python packages >* a dependency on numpy >* a dependency on elementtree for Python < 2.5 >* textbook pdfs to be installed centrally >* 200Mb of data (uncompressed) to be installed centrally The above are relatively easy; just put the data and docs inside of your package directories, add install_package_data=True and install_requires=['numpy','elementtree'] (more or less) and import everything from setuptools instead of distutils.core. >* similar amount of data to be generated at install time and stored >centrally ("models") >* directory tree to be stored to user filespace (code samples from >the textbook) These two aren't currently supported. The closest you can get is to include scripts that users can run to generate, install, copy, etc., as eggs are simply installed as-is under normal circumstances, and to a single location. >* locations of all data to be discovered at run-time by the toolkit Use the pkg_resources API calls to do this, and you're set. From vivainio at gmail.com Thu Oct 19 22:01:59 2006 From: vivainio at gmail.com (Ville M. Vainio) Date: Thu, 19 Oct 2006 23:01:59 +0300 Subject: [Distutils] easier_install Message-ID: <46cb515a0610191301x19f2a1b3k3bbdae264588ba07@mail.gmail.com> Someone here might have use for this (a trivial "wizard" for creating setup.py files): http://cheeseshop.python.org/pypi/easier_install/0.1 -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From ianb at colorstudy.com Fri Oct 20 17:56:57 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Fri, 20 Oct 2006 10:56:57 -0500 Subject: [Distutils] distutils doc page Message-ID: <4538F1C9.90608@colorstudy.com> I was just noticing that the current distutils page (http://python.org/doc/current/lib/module-distutils.html) doesn't have any mention of setuptools or any modern packaging stuff. I think we should suggest that some links be added to that page. But in terms of creating a Setuptools package, I'm not sure what the best tutorial would be (I think the Setuptools docs are a bit more in-depth, and a link to a simpler and less comprehensive tutorial would also be good there). Suggestions for a tutorial to link to? -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From JRBoverhof at lbl.gov Sun Oct 22 01:24:06 2006 From: JRBoverhof at lbl.gov (Joshua Boverhof) Date: Sat, 21 Oct 2006 16:24:06 -0700 Subject: [Distutils] SandboxViolation: open('/dev/urandom', 0) {} Message-ID: 4Suite is listed as one of my project's setuptools "install_requires", but when I attempt to install on Mac OSX-10.4.8 using python-2.5 I get an error: """ Reading http://www.python.org/pypi/4Suite-XML/1.0 Best match: 4Suite-XML 1.0 Downloading http://cheeseshop.python.org/packages/source/4/4Suite-XML/ 4Suite-XML-1.0.zip#md5=0b61c214cd03ecb2f673b52b316c0a84 Processing 4Suite-XML-1.0.zip Running 4Suite-XML-1.0/setup.py -q bdist_egg --dist-dir /tmp/ easy_install-_X4a0x/4Suite-XML-1.0/egg-dist-tmp-uBG5YU error: Setup script exited with error: SandboxViolation: open('/dev/ urandom', 0) {} The package setup script has attempted to modify files on your system that are not within the EasyInstall build area, and has been aborted. This package cannot be safely installed by EasyInstall, and may not support alternate installation locations even if you run its setup script by hand. Please inform the package's author and the EasyInstall maintainers to find out if a fix or workaround is available. """ So setuptools isn't allowing 4Suite to access the random number generator during build/installation. I can easily subvert this, in case anyone else just wants to make this work: """ def main(): from setuptools import sandbox sandbox.DirectorySandbox._ok = lambda self,path: True """ Of course I'd like a better way of doing this, maybe I missed something in the setuptools docs on configuration? -josh -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2526 bytes Desc: not available Url : http://mail.python.org/pipermail/distutils-sig/attachments/20061021/62b3a9b5/attachment.bin From bob at redivi.com Sun Oct 22 01:39:17 2006 From: bob at redivi.com (Bob Ippolito) Date: Sat, 21 Oct 2006 16:39:17 -0700 Subject: [Distutils] SandboxViolation: open('/dev/urandom', 0) {} In-Reply-To: References: Message-ID: <6a36e7290610211639o19d20fb8r602e2bad6d90f894@mail.gmail.com> On 10/21/06, Joshua Boverhof wrote: > 4Suite is listed as one of my project's setuptools > "install_requires", but when I attempt to install on Mac OSX-10.4.8 > using python-2.5 I get an error: > > > """ > Reading http://www.python.org/pypi/4Suite-XML/1.0 > Best match: 4Suite-XML 1.0 > Downloading http://cheeseshop.python.org/packages/source/4/4Suite-XML/ > 4Suite-XML-1.0.zip#md5=0b61c214cd03ecb2f673b52b316c0a84 > Processing 4Suite-XML-1.0.zip > Running 4Suite-XML-1.0/setup.py -q bdist_egg --dist-dir /tmp/ > easy_install-_X4a0x/4Suite-XML-1.0/egg-dist-tmp-uBG5YU > error: Setup script exited with error: SandboxViolation: open('/dev/ > urandom', 0) {} > > The package setup script has attempted to modify files on your system > that are not within the EasyInstall build area, and has been aborted. > > This package cannot be safely installed by EasyInstall, and may not > support alternate installation locations even if you run its setup > script by hand. Please inform the package's author and the EasyInstall > maintainers to find out if a fix or workaround is available. > """ > > > So setuptools isn't allowing 4Suite to access the random number > generator during build/installation. I can easily subvert this, in > case anyone else just wants to make this work: > > """ > def main(): > from setuptools import sandbox > sandbox.DirectorySandbox._ok = lambda self,path: True > > """ > > Of course I'd like a better way of doing this, maybe I missed > something in the setuptools docs on configuration? WTF is a build script doing with a random number generator?! -bob From jeremy.kloth at 4suite.org Sun Oct 22 02:36:11 2006 From: jeremy.kloth at 4suite.org (Jeremy Kloth) Date: Sat, 21 Oct 2006 18:36:11 -0600 Subject: [Distutils] [4suite] Re: SandboxViolation: open('/dev/urandom', 0) {} In-Reply-To: <6a36e7290610211639o19d20fb8r602e2bad6d90f894@mail.gmail.com> References: <6a36e7290610211639o19d20fb8r602e2bad6d90f894@mail.gmail.com> Message-ID: <200610211836.11652.jeremy.kloth@4suite.org> On Saturday 21 October 2006 5:39 pm, Bob Ippolito wrote: > On 10/21/06, Joshua Boverhof wrote: > > 4Suite is listed as one of my project's setuptools > > "install_requires", but when I attempt to install on Mac OSX-10.4.8 > > using python-2.5 I get an error: > > > > > > """ > > Reading http://www.python.org/pypi/4Suite-XML/1.0 > > Best match: 4Suite-XML 1.0 > > Downloading http://cheeseshop.python.org/packages/source/4/4Suite-XML/ > > 4Suite-XML-1.0.zip#md5=0b61c214cd03ecb2f673b52b316c0a84 > > Processing 4Suite-XML-1.0.zip > > Running 4Suite-XML-1.0/setup.py -q bdist_egg --dist-dir /tmp/ > > easy_install-_X4a0x/4Suite-XML-1.0/egg-dist-tmp-uBG5YU > > error: Setup script exited with error: SandboxViolation: open('/dev/ > > urandom', 0) {} > > > > The package setup script has attempted to modify files on your system > > that are not within the EasyInstall build area, and has been aborted. > > > > This package cannot be safely installed by EasyInstall, and may not > > support alternate installation locations even if you run its setup > > script by hand. Please inform the package's author and the EasyInstall > > maintainers to find out if a fix or workaround is available. > > """ This is a bug in setuptools' sandboxing. This is os.urandom() using *os.open* which uses integer constants not character constants. > > So setuptools isn't allowing 4Suite to access the random number > > generator during build/installation. I can easily subvert this, in > > case anyone else just wants to make this work: > > > > """ > > def main(): > > from setuptools import sandbox > > sandbox.DirectorySandbox._ok = lambda self,path: True > > > > """ > > > > Of course I'd like a better way of doing this, maybe I missed > > something in the setuptools docs on configuration? > > WTF is a build script doing with a random number generator?! Generating UUIDs. Either way, I'll see what can be done to eliminate the need to generate the UUID in the first place. -- Jeremy Kloth http://4suite.org/ From jim at zope.com Sun Oct 22 19:52:24 2006 From: jim at zope.com (Jim Fulton) Date: Sun, 22 Oct 2006 13:52:24 -0400 Subject: [Distutils] YAGNI extras and tests_require Message-ID: <453BAFD8.6060200@zope.com> I'd like to call "Ya aint gonna need it" on the extras feature of setuptools. As far as I can tell, extras are just a way to avoid fine-grained packages. Is this benefit worth the complexity? I don't think so. It violates "There's Only One Way To Do It" and increases the complexity of setuptools. Setuptools is wonderful but it is complex. I think it would be helpful to make it simpler and I really don't see a need for extras. I think a similar argument could be made against the tests_require feature. (In the presence of the extras feature, it's puzzling that this isn't handled as an extra.) Jim -- Jim Fulton mailto:jim at zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From matt at matt-good.net Sun Oct 22 23:15:29 2006 From: matt at matt-good.net (Matt Good) Date: Sun, 22 Oct 2006 17:15:29 -0400 Subject: [Distutils] YAGNI extras and tests_require In-Reply-To: <453BBC2C.3050105@zope.com> References: <453BAFD8.6060200@zope.com> <1161542433.5942.16.camel@nny> <453BBC2C.3050105@zope.com> Message-ID: <1161551729.19139.24.camel@nny> Oops, I forgot to copy the list on my first respose. On Sun, 2006-10-22 at 14:45 -0400, Jim Fulton wrote: > Matt Good wrote: > > On Sun, 2006-10-22 at 13:52 -0400, Jim Fulton wrote: > >> I'd like to call "Ya aint gonna need it" on the extras feature of setuptools. > >> > >> As far as I can tell, extras are just a way to avoid fine-grained packages. > >> Is this benefit worth the complexity? I don't think so. It violates > >> "There's Only One Way To Do It" and increases the complexity of setuptools. > >> > >> Setuptools is wonderful but it is complex. I think it would be helpful to > >> make it simpler and I really don't see a need for extras. > > > > Well, I've found them to be useful with entry points in Trac plugins. > > For example the SpamFilter plugin provides several spam filtering > > methods, not all of which will be needed by every user. There's an IP > > blacklist filter that requires DNSPython. By declaring the entry point > > for the blacklist module with an extra for DNSPython the module will > > only load if that requirement is available. > > Why not just distribute the IP blacklist filter as a separate package? > > > > > I don't think it's uncommon for packages to have optional features that > > require additional dependencies, and extras provide an easy way to > > declare those dependencies. Sure, sometimes these features could be > > broken out into a separate package, but in the cases where I've used > > extras making additional packages would be the more complex solution. > > How more complex? You would simply have had more packages? Is that > really more complex that extras? Yes. An extra is simply a line or two in setup.py. A separate package means another branch in SVN, more releases to make, upload to the Cheeseshop, more files for users to download, and unneeded duplication between the original and new package. This seems like an awful lot of additional work just to justify removing a feature from setuptools. > Is it advantagous enough to violate TWOOWTDI? How does it violate that? There's *always* more than one way, but there should be one clear "right way", and I think extras *is* that way. There are clearly times when the right thing is to create a new separate package, but I think that new package should have sufficient substance and be decoupled from the original to justify distributing it separately. > Is it really worth this extra complexity in setuptools? Well, since Phillip is maintaining it I guess it's his decision. I think it's feature that's useful to developers, so I'm in favor of keeping it. -- Matt Good From jim at zope.com Mon Oct 23 00:27:07 2006 From: jim at zope.com (Jim Fulton) Date: Sun, 22 Oct 2006 18:27:07 -0400 Subject: [Distutils] YAGNI extras and tests_require In-Reply-To: <1161551729.19139.24.camel@nny> References: <453BAFD8.6060200@zope.com> <1161542433.5942.16.camel@nny> <453BBC2C.3050105@zope.com> <1161551729.19139.24.camel@nny> Message-ID: <453BF03B.3080208@zope.com> Matt Good wrote: > Oops, I forgot to copy the list on my first respose. > > On Sun, 2006-10-22 at 14:45 -0400, Jim Fulton wrote: >> Matt Good wrote: >>> On Sun, 2006-10-22 at 13:52 -0400, Jim Fulton wrote: >>>> I'd like to call "Ya aint gonna need it" on the extras feature of setuptools. >>>> >>>> As far as I can tell, extras are just a way to avoid fine-grained packages. >>>> Is this benefit worth the complexity? I don't think so. It violates >>>> "There's Only One Way To Do It" and increases the complexity of setuptools. >>>> >>>> Setuptools is wonderful but it is complex. I think it would be helpful to >>>> make it simpler and I really don't see a need for extras. >>> Well, I've found them to be useful with entry points in Trac plugins. >>> For example the SpamFilter plugin provides several spam filtering >>> methods, not all of which will be needed by every user. There's an IP >>> blacklist filter that requires DNSPython. By declaring the entry point >>> for the blacklist module with an extra for DNSPython the module will >>> only load if that requirement is available. >> Why not just distribute the IP blacklist filter as a separate package? >> >>> I don't think it's uncommon for packages to have optional features that >>> require additional dependencies, and extras provide an easy way to >>> declare those dependencies. Sure, sometimes these features could be >>> broken out into a separate package, but in the cases where I've used >>> extras making additional packages would be the more complex solution. >> How more complex? You would simply have had more packages? Is that >> really more complex that extras? > > Yes. An extra is simply a line or two in setup.py. A separate package > means another branch in SVN, Not necessarily. You could manage multiple related packages as a single SVN project. I do. > more releases to make, upload to the > Cheeseshop, True. > more files for users to download, If the downloads are automated, does this make a difference? > and unneeded duplication > between the original and new package. Why duplication? Likely, one of the packages would simply depend on the other. > This seems like an awful lot of > additional work just to justify removing a feature from setuptools. I think the extra work is just making the extra releases, at least assuming automated downloads. >> Is it advantagous enough to violate TWOOWTDI? > > How does it violate that? There's *always* more than one way, but there > should be one clear "right way", and I think extras *is* that way. There don't have to be multiple ways. I also don't see extras as being clearly superior, but we disagree. Jim -- Jim Fulton mailto:jim at zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From jorge.vargas at gmail.com Mon Oct 23 00:42:46 2006 From: jorge.vargas at gmail.com (Jorge Vargas) Date: Sun, 22 Oct 2006 18:42:46 -0400 Subject: [Distutils] YAGNI extras and tests_require In-Reply-To: <453BAFD8.6060200@zope.com> References: <453BAFD8.6060200@zope.com> Message-ID: <32822fe60610221542p4f0fdaafsfa330a0777594c89@mail.gmail.com> On 10/22/06, Jim Fulton wrote: > > I'd like to call "Ya aint gonna need it" on the extras feature of setuptools. > actually I find it as a very interesting way of implementing a plugin architecture. in fact I was thinking of trying it out with a project I'm working on like an hour ago. I have 2 main objects Sources and Bots (aka interfaces) and then I have Sources as in MysqldbSource ,SQLObjectSource, SQLAlchemySource,etc. and Bots as in JabberBot, GoogleBot, MSNBot. Also eventually people will subclass std Sources to get their own, set of data "publish" on the system As you can see a "basic" instalation of this will be at least 1 bot and 1 source then everything else will get installed as optional packages. > As far as I can tell, extras are just a way to avoid fine-grained packages. > Is this benefit worth the complexity? I don't think so. It violates > "There's Only One Way To Do It" and increases the complexity of setuptools. > > Setuptools is wonderful but it is complex. I think it would be helpful to > make it simpler and I really don't see a need for extras. > I aggree on that, I have been around it for some time now and I'm still confused :) > I think a similar argument could be made against the tests_require feature. > (In the presence of the extras feature, it's puzzling that this isn't > handled as an extra.) > I have never use/seen that one. > Jim > > -- > Jim Fulton mailto:jim at zope.com Python Powered! > CTO (540) 361-1714 http://www.python.org > Zope Corporation http://www.zope.com http://www.zope.org > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > From ianb at colorstudy.com Mon Oct 23 03:33:38 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Sun, 22 Oct 2006 20:33:38 -0500 Subject: [Distutils] YAGNI extras and tests_require In-Reply-To: <453BAFD8.6060200@zope.com> References: <453BAFD8.6060200@zope.com> Message-ID: <453C1BF2.8@colorstudy.com> Jim Fulton wrote: > I'd like to call "Ya aint gonna need it" on the extras feature of setuptools. > > As far as I can tell, extras are just a way to avoid fine-grained packages. > Is this benefit worth the complexity? I don't think so. It violates > "There's Only One Way To Do It" and increases the complexity of setuptools. FWIW, I find it too difficult to communicate what extras mean, so I haven't used them. If something is really optional, then I just make it optional, and if you want to use that option you have to require the necessary package. Otherwise it's easier to do package refactoring to keep the dependencies straight-forward. For tests I'd rather just produce errors that are useful, and let people figure out from there that they need to add another package. Part of why I find the requirements system hard to depend on is that the errors are very hard to understand, so I'm inclined to keep the requirements fairly course-grained. > Setuptools is wonderful but it is complex. I think it would be helpful to > make it simpler and I really don't see a need for extras. > > I think a similar argument could be made against the tests_require feature. > (In the presence of the extras feature, it's puzzling that this isn't > handled as an extra.) A reluctance to make a specific extra name as special? Otherwise yes, it seems clear enough to just make it an extra. -- Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org From pje at telecommunity.com Mon Oct 23 05:13:23 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 22 Oct 2006 20:13:23 -0700 Subject: [Distutils] YAGNI extras and tests_require In-Reply-To: <453BAFD8.6060200@zope.com> References: <453BAFD8.6060200@zope.com> Message-ID: <7.0.1.0.0.20061022200721.0210b6e0@telecommunity.com> At 10:52 AM 10/22/2006, Jim Fulton wrote: >I'd like to call "Ya aint gonna need it" on the extras feature of setuptools. > >As far as I can tell, extras are just a way to avoid fine-grained packages. >Is this benefit worth the complexity? I don't think so. It violates >"There's Only One Way To Do It" and increases the complexity of setuptools. > >Setuptools is wonderful but it is complex. I think it would be helpful to >make it simpler and I really don't see a need for extras. > >I think a similar argument could be made against the tests_require feature. >(In the presence of the extras feature, it's puzzling that this isn't >handled as an extra.) tests_require and setup_requires don't get installed; extras are. Also, the mechanism involved is different; you have to have a valid distribution object in order to reference its extras, and when tests_require is being processed, you may not have that yet. (I don't know without looking at the code, and I'm on the road right now.) Anyway, I don't see tests_require as having any relationship to extras; many packages don't have installable tests, for example. From rasky at develer.com Mon Oct 23 17:11:28 2006 From: rasky at develer.com (Giovanni Bajo) Date: Mon, 23 Oct 2006 17:11:28 +0200 Subject: [Distutils] Problem with ihooks: not working as expected in Py25 with eggs? Message-ID: <01d701c6f6b5$8391c600$e303030a@trilan> Hello, I'm using the following recipe (which I shall publish one day in the cookbook): ============== nobarepyc.py ========================= import ihooks import os class _NoBarePycHooks(ihooks.Hooks): def load_compiled(self, name, filename, *args, **kwargs): sourcefn = os.path.splitext(filename)[0] + ".py" if not os.path.isfile(sourcefn): raise ImportError('forbidden import of bare .pyc file: %r' % filename) return ihooks.Hooks.load_compiled(name, filename, *args, **kwargs) ihooks.ModuleImporter(ihooks.ModuleLoader(_NoBarePycHooks())).install() ============== nobarepyc.py ========================= This is useful when managing Python source trees with CVS/SVN: after a file is renamed or moved, you don't want imports to silently import the previously generated .pyc file. By importing this module in debug builds, you install an import hook that makes so a "bare" .pyc file (with no .py file next to it) is never imported. Now, this file does not seem to work with packages shipped as .egg files, at least in Python 2.5. Specifically, I get an import error with setuptools itself: >>> from misc import nobarepyc >>> import pkg_resources Traceback (most recent call last): File "", line 1, in File "C:\Python25\lib\ihooks.py", line 404, in import_module q, tail = self.find_head_package(parent, str(name)) File "C:\Python25\lib\ihooks.py", line 447, in find_head_package raise ImportError, "No module named " + qname ImportError: No module named pkg_resources *IF* I can read this traceback correctly, this has nothing to do with my own hook: it's ihooks itself which appears to be broken with .egg files. Is that correct? Does .egg files break ihooks? What is the correct way of writing this sort of import hooks with setuptools / .egg files? Thanks -- Giovanni Bajo From e-huss at netmeridian.com Mon Oct 23 21:04:17 2006 From: e-huss at netmeridian.com (Eric Huss) Date: Mon, 23 Oct 2006 12:04:17 -0700 (PDT) Subject: [Distutils] clean and swig/pyrex Message-ID: I've noticed that the "clean" command does not remove files generated by swig or pyrex (since pyrex uses the same swig interface). Does anyone have any suggestions on how to make that work, or a suggestion on how to enhance distutils to make it work? -Eric From pje at telecommunity.com Tue Oct 24 00:12:59 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 23 Oct 2006 18:12:59 -0400 Subject: [Distutils] Problem with ihooks: not working as expected in Py25 with eggs? In-Reply-To: <01d701c6f6b5$8391c600$e303030a@trilan> Message-ID: <5.1.1.6.0.20061023180808.02f55be0@sparrow.telecommunity.com> At 05:11 PM 10/23/2006 +0200, Giovanni Bajo wrote: > >>> from misc import nobarepyc > >>> import pkg_resources >Traceback (most recent call last): > File "", line 1, in > File "C:\Python25\lib\ihooks.py", line 404, in import_module > q, tail = self.find_head_package(parent, str(name)) > File "C:\Python25\lib\ihooks.py", line 447, in find_head_package > raise ImportError, "No module named " + qname >ImportError: No module named pkg_resources > >*IF* I can read this traceback correctly, this has nothing to do with my own >hook: it's ihooks itself which appears to be broken with .egg files. Is that >correct? Does .egg files break ihooks? If I'm reading it correctly, ihooks expects modules to be files, period. So it's not going to work with any zipfile, let alone eggs. The workaround would be to configure easy_install to --always-unzip when installing. See the manual for details. (Note that ihooks predates PEP 302 and therefore does not support it. ihooks in fact *replaces* the built-in import mechanism entirely, so it's not likely to work correctly with any other PEP 302-based features besides eggs.) From rasky at develer.com Tue Oct 24 00:25:44 2006 From: rasky at develer.com (Giovanni Bajo) Date: Tue, 24 Oct 2006 00:25:44 +0200 Subject: [Distutils] Problem with ihooks: not working as expected in Py25 with eggs? References: <5.1.1.6.0.20061023180808.02f55be0@sparrow.telecommunity.com> Message-ID: <09e401c6f6f2$2e5356d0$e303030a@trilan> Phillip J. Eby wrote: > If I'm reading it correctly, ihooks expects modules to be files, > period. So it's not going to work with any zipfile, let alone eggs. > The workaround would be to configure easy_install to --always-unzip > when installing. See the manual for details. Yes that's a workaround but I'd rather avoid it if possible. > (Note that ihooks predates PEP 302 and therefore does not support > it. ihooks in fact *replaces* the built-in import mechanism > entirely, so it's not likely to work correctly with any other PEP > 302-based features besides eggs.) I see. If you were to write the module I showed you, using a PEP-302 compatible system, what would you use? Is it possible to achieve the same thing in such few lines, but being PEP-302 compatible? -- Giovanni Bajo From tseaver at palladion.com Tue Oct 24 01:18:16 2006 From: tseaver at palladion.com (Tres Seaver) Date: Mon, 23 Oct 2006 19:18:16 -0400 Subject: [Distutils] clean and swig/pyrex In-Reply-To: References: Message-ID: <453D4DB8.5020203@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Eric Huss wrote: > I've noticed that the "clean" command does not remove files generated by > swig or pyrex (since pyrex uses the same swig interface). Does anyone > have any suggestions on how to make that work, or a suggestion on how to > enhance distutils to make it work? Shouldn't such "generated sources" be removed only by something like 'distclean'? Otherwise, it becomse unsafe to distribute except to those who have swig / pyrex themselves to do the generation. Tres. - -- =================================================================== Tres Seaver +1 202-558-7113 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFFPU24+gerLs4ltQ4RAslOAKCQjS6qwov3jAknJxUzrxDcut/dkgCdH/jD gQ5CqE3chiav7N8lxIJmU14= =NWd/ -----END PGP SIGNATURE----- From pje at telecommunity.com Tue Oct 24 01:19:27 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 23 Oct 2006 19:19:27 -0400 Subject: [Distutils] Problem with ihooks: not working as expected in Py25 with eggs? In-Reply-To: <09e401c6f6f2$2e5356d0$e303030a@trilan> References: <5.1.1.6.0.20061023180808.02f55be0@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20061023191710.026cc008@sparrow.telecommunity.com> At 12:25 AM 10/24/2006 +0200, Giovanni Bajo wrote: >Phillip J. Eby wrote: > > > If I'm reading it correctly, ihooks expects modules to be files, > > period. So it's not going to work with any zipfile, let alone eggs. > > The workaround would be to configure easy_install to --always-unzip > > when installing. See the manual for details. > >Yes that's a workaround but I'd rather avoid it if possible. > > > (Note that ihooks predates PEP 302 and therefore does not support > > it. ihooks in fact *replaces* the built-in import mechanism > > entirely, so it's not likely to work correctly with any other PEP > > 302-based features besides eggs.) > >I see. If you were to write the module I showed you, using a PEP-302 >compatible system, what would you use? Is it possible to achieve the same >thing in such few lines, but being PEP-302 compatible? Not in just a few lines, no. It would take quite a few, I'm afraid. PEP 302 doesn't provide a filesystem abstraction, and isn't likely to grow one soon. Maybe by Python 2.6 or 3.0, depending on how much free time I end up getting in the next couple of years. :) From pje at telecommunity.com Tue Oct 24 01:21:52 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 23 Oct 2006 19:21:52 -0400 Subject: [Distutils] setuptools: but with svn-tag-revision and uncommitted project In-Reply-To: <4536A7CA.8040201@colorstudy.com> References: <4523FCAE.8000208@colorstudy.com> <4523FCAE.8000208@colorstudy.com> Message-ID: <5.1.1.6.0.20061023192009.02be4090@sparrow.telecommunity.com> This problem was fixed in r52006 and r52007 of the setuptools trunk and 0.6 branch, respectively. But the fix isn't available in any released varsion at the moment. At 05:16 PM 10/18/2006 -0500, Ian Bicking wrote: >Anything on this? > >Ian Bicking wrote: > > I get this exception when I run "python setup.py egg_info" with > > --tag-svn-revision and when the project has never been committed (and > > hence there's no committed-rev). I think it should instead give 0, or > > at least a good error message. (I don't see any problem with 0) > > > > Traceback (most recent call last): > > File "setup.py", line 24, in ? > > entry_points=""" > > File "/usr/lib/python2.4/distutils/core.py", line 149, in setup > > dist.run_commands() > > File "/usr/lib/python2.4/distutils/dist.py", line 946, in run_commands > > self.run_command(cmd) > > File "/usr/lib/python2.4/distutils/dist.py", line 965, in run_command > > cmd_obj.ensure_finalized() > > File "/usr/lib/python2.4/distutils/cmd.py", line 117, in > ensure_finalized > > self.finalize_options() > > File > > > "/usr/local/lib/python2.4/site-packages/setuptools-0.6c3-py2.4.egg/setuptools/command/egg_info.py", > > > line 85, in finalize_options > > self.vtags = self.tags() > > File > > > "/usr/local/lib/python2.4/site-packages/setuptools-0.6c3-py2.4.egg/setuptools/command/egg_info.py", > > > line 179, in tags > > ): version += '-r%s' % self.get_svn_revision() > > File > > > "/usr/local/lib/python2.4/site-packages/setuptools-0.6c3-py2.4.egg/setuptools/command/egg_info.py", > > > line 227, in get_svn_revision > > localrev = max([int(m.group(1)) for m in revre.finditer(data)]) > > ValueError: max() arg is an empty sequence > > > >_______________________________________________ >Distutils-SIG maillist - Distutils-SIG at python.org >http://mail.python.org/mailman/listinfo/distutils-sig From pje at telecommunity.com Tue Oct 24 20:38:27 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 24 Oct 2006 14:38:27 -0400 Subject: [Distutils] [4suite] Re: SandboxViolation: open('/dev/urandom', 0) {} In-Reply-To: <200610211836.11652.jeremy.kloth@4suite.org> References: <6a36e7290610211639o19d20fb8r602e2bad6d90f894@mail.gmail.com> <6a36e7290610211639o19d20fb8r602e2bad6d90f894@mail.gmail.com> Message-ID: <5.1.1.6.0.20061024143714.03a76d48@sparrow.telecommunity.com> At 06:36 PM 10/21/2006 -0600, Jeremy Kloth wrote: >On Saturday 21 October 2006 5:39 pm, Bob Ippolito wrote: > > On 10/21/06, Joshua Boverhof wrote: > > > 4Suite is listed as one of my project's setuptools > > > "install_requires", but when I attempt to install on Mac OSX-10.4.8 > > > using python-2.5 I get an error: > > > > > > > > > """ > > > Reading http://www.python.org/pypi/4Suite-XML/1.0 > > > Best match: 4Suite-XML 1.0 > > > Downloading http://cheeseshop.python.org/packages/source/4/4Suite-XML/ > > > 4Suite-XML-1.0.zip#md5=0b61c214cd03ecb2f673b52b316c0a84 > > > Processing 4Suite-XML-1.0.zip > > > Running 4Suite-XML-1.0/setup.py -q bdist_egg --dist-dir /tmp/ > > > easy_install-_X4a0x/4Suite-XML-1.0/egg-dist-tmp-uBG5YU > > > error: Setup script exited with error: SandboxViolation: open('/dev/ > > > urandom', 0) {} > > > > > > The package setup script has attempted to modify files on your system > > > that are not within the EasyInstall build area, and has been aborted. > > > > > > This package cannot be safely installed by EasyInstall, and may not > > > support alternate installation locations even if you run its setup > > > script by hand. Please inform the package's author and the EasyInstall > > > maintainers to find out if a fix or workaround is available. > > > """ > >This is a bug in setuptools' sandboxing. This is os.urandom() using >*os.open* >which uses integer constants not character constants. Actually, that's not the issue; it's just that os.open was prohibited from accessing any paths outside the sandbox, regardless of flags. I've fixed this in the SVN version now, and it'll go out in the 0.6c4 release. From pje at telecommunity.com Tue Oct 24 20:50:34 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 24 Oct 2006 14:50:34 -0400 Subject: [Distutils] easy_install --always-copy and setup.py develop In-Reply-To: <452B0738.7060006@colorstudy.com> Message-ID: <5.1.1.6.0.20061024144649.041c7318@sparrow.telecommunity.com> At 09:36 PM 10/9/2006 -0500, Ian Bicking wrote: >I have a package (PasteDeploy) installed with "python setup.py develop", >at version 0.9.7dev-r5544. There's a released version 0.9.6. When I do >"easy_install --always-copy --install-dir /some/dir >SomePackageThatRequiresPasteDeploy" it gives me errors like error: > >Could not find suitable distribution for >Requirement.parse('PasteDeploy==0.9.7dev-r5544') (--always-copy skips >system and development eggs) > >There is no requirement like that, it seems to be coming up with that >version specifier on its own. --always-copy currently tries to copy every distribution that was used to resolve dependencies for the explicitly requested installation target. I don't know of any way to fix this right off; unless somebody else has an idea it'll have to be left alone for now. :( From antoine.pitrou at wengo.fr Wed Oct 25 12:28:53 2006 From: antoine.pitrou at wengo.fr (Antoine Pitrou) Date: Wed, 25 Oct 2006 12:28:53 +0200 Subject: [Distutils] setting the right owner of local subdirectories Message-ID: <1161772134.5921.6.camel@antoine-ubuntu> Hi, Under Unix-alikes, you have to be root (thus using "sudo" or "su") in order to truly install a package (even with the "develop" command since it copies some info into the system directories). But an annoying side-effect is that, when you first run "sudo python setup.py develop" (or "install"), it also creates directories in the local directory under the root user. Which means that, when you later run commands that don't need to be root to be issued, you get the following kinds of error: $ ./setup.py test running test running egg_info writing flapflap.egg-info/PKG-INFO writing top-level names to flapflap.egg-info/top_level.txt writing dependency_links to flapflap.egg-info/dependency_links.txt writing entry points to flapflap.egg-info/entry_points.txt error: flapflap.egg-info/entry_points.txt: Permission denied It is a minor but repetitive annoyance. It would be nicer if, when creating those local subdirectories, setup.py would try to change the owner to the owner of the current parent directory (the project base directory). Regards Antoine. From bignose+hates-spam at benfinney.id.au Wed Oct 25 14:05:36 2006 From: bignose+hates-spam at benfinney.id.au (Ben Finney) Date: Wed, 25 Oct 2006 22:05:36 +1000 Subject: [Distutils] setting the right owner of local subdirectories References: <1161772134.5921.6.camel@antoine-ubuntu> Message-ID: <87u01smwhb.fsf@benfinney.id.au> Antoine Pitrou writes: > But an annoying side-effect is that, when you first run "sudo python > setup.py develop" (or "install"), it also creates directories in the > local directory under the root user. You can configure sudo to set HOME to the home directory of the target user. To do this every time the target is 'root', edit sudoers and add this line: Defaults>root always_set_home Read the sudoers(5) man page for more. -- \ "If consumers even know there's a DRM, what it is, and how it | `\ works, we've already failed." -- Peter Lee, Disney | _o__) corporation, 2005 | Ben Finney From antoine.pitrou at wengo.fr Wed Oct 25 16:02:02 2006 From: antoine.pitrou at wengo.fr (Antoine Pitrou) Date: Wed, 25 Oct 2006 16:02:02 +0200 Subject: [Distutils] setting the right owner of local subdirectories In-Reply-To: <87u01smwhb.fsf@benfinney.id.au> References: <1161772134.5921.6.camel@antoine-ubuntu> <87u01smwhb.fsf@benfinney.id.au> Message-ID: <1161784922.5921.10.camel@antoine-ubuntu> Le mercredi 25 octobre 2006 ? 22:05 +1000, Ben Finney a ?crit : > Antoine Pitrou writes: > > > But an annoying side-effect is that, when you first run "sudo python > > setup.py develop" (or "install"), it also creates directories in the > > local directory under the root user. > > You can configure sudo to set HOME to the home directory of the target > user. To do this every time the target is 'root', edit sudoers and add > this line: Sorry, my wording was misleading. I meant that setuptools creates subdirectories inside the local project directory (which is fine), and if it was invoked with "sudo" those subdirectories are owned by the root user (which is less fine). From matt at matt-good.net Wed Oct 25 17:49:00 2006 From: matt at matt-good.net (Matt Good) Date: Wed, 25 Oct 2006 11:49:00 -0400 Subject: [Distutils] setting the right owner of local subdirectories In-Reply-To: <1161772134.5921.6.camel@antoine-ubuntu> References: <1161772134.5921.6.camel@antoine-ubuntu> Message-ID: <1161791340.26627.12.camel@nny> Oops, forgot to copy the list again. On Wed, 2006-10-25 at 12:28 +0200, Antoine Pitrou wrote: > Under Unix-alikes, you have to be root (thus using "sudo" or "su") in > order to truly install a package (even with the "develop" command since > it copies some info into the system directories). > > But an annoying side-effect is that, when you first run "sudo python > setup.py develop" (or "install"), it also creates directories in the > local directory under the root user. > > Which means that, when you later run commands that don't need to be root > to be issued, you get the following kinds of error: > > $ ./setup.py test > running test > running egg_info > writing flapflap.egg-info/PKG-INFO > writing top-level names to flapflap.egg-info/top_level.txt > writing dependency_links to flapflap.egg-info/dependency_links.txt > writing entry points to flapflap.egg-info/entry_points.txt > error: flapflap.egg-info/entry_points.txt: Permission denied > > It is a minor but repetitive annoyance. It would be nicer if, when > creating those local subdirectories, setup.py would try to change the > owner to the owner of the current parent directory (the project base > directory). With "make" you normally use two commands to separate the build from the install: $ make $ sudo make install With setuptools the working combination seems to be: $ ./setup.py egg_info $ sudo ./setup.py develop Or: $ ./setup.py bdist_egg $ sudo ./setup.py install I wouldn't mind seeing setuptools be smarter about this, but those workarounds have done the trick for me. -- Matt Good From chris at wxnet.org Wed Oct 25 20:05:00 2006 From: chris at wxnet.org (Christopher Blunck) Date: Wed, 25 Oct 2006 14:05:00 -0400 Subject: [Distutils] bdist_rpm fails due to bytecompiled modules Message-ID: <20061025180500.GA27367@night.wxnet.org> hello, i have a very basic pure-python package and script that i would like to distribute as a noarch rpm using distutils. i am using FC5 as my development platform and have prepared a setup.py that properly builds and installs my application. the first problem i have is that the bdist_rpm command results in an %install section as follows: %install python setup.py install --single-version-externally-managed --root=$RPM_BUILD_ROOT --record=INSTALLED_FILES the trouble here with that %install section is that the python modules get bytecompiled and thus turned into native code. i'd prefer for that not to happen, so i have written my own install.sh script and i provide that to my python setup.py call as follows: $ python setup.py bdist_rpm --spec-only --install-script /path/to/install.sh my install.sh passes the --optimize 0 and --no-compile arguments to distutils, resulting in no .pyc or .pyo files being installed in my RPM_BUILD_ROOT. the contents of my install.sh are as follows: python setup.py install --single-version-externally-managed --root=$RPM_BUILD_ROOT --record=INSTALLED_FILES --optimize 0 --no-compile i thought that would result in no byte compiled files but unfortunately i was wrong. i'm not sure what's causing it, but rpmbuild ends up calling brp-python-bytecompile to compile the .py files. i noticed in the ChangeLog for rpm the following: - force *.py->*.pyo byte code compilation with brp-python-bytecompile. so it appears to me as tho there is no way to assemble a noarch RPM using distutils. does that sound accurate to you all? or are those folks out there using distutils and an old version of rpm (prior to 2005) that are able to assemble noarch RPMs? -c -- WeatherNet Observations for station: home Temperature: 54.80F Pressure: 30.02in; Dew Point: 30.76F (40%) Wind: 248 at 5 mph Recorded: 17:54:59 10/25/06 (http://wsdl.wxnet.org/inquiry/binding.wsdl) From ronaldoussoren at mac.com Wed Oct 25 21:01:03 2006 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Wed, 25 Oct 2006 21:01:03 +0200 Subject: [Distutils] bdist_rpm fails due to bytecompiled modules In-Reply-To: <20061025180500.GA27367@night.wxnet.org> References: <20061025180500.GA27367@night.wxnet.org> Message-ID: <0D8C569F-5A77-4909-A153-24A083E47C18@mac.com> On Oct 25, 2006, at 8:05 PM, Christopher Blunck wrote: > > > the trouble here with that %install section is that the python modules > get bytecompiled and thus turned into native code. i'd prefer for > that not to happen, so i have written my own install.sh script and i > provide that to my python setup.py call as follows: .pyc and .pyo files aren't native code, they are bytecode files that work on all platforms. There should therefore be no need to surpress bytecode compilation. Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3562 bytes Desc: not available Url : http://mail.python.org/pipermail/distutils-sig/attachments/20061025/b78aec5b/attachment.bin From chris at wxnet.org Wed Oct 25 21:13:19 2006 From: chris at wxnet.org (Christopher Blunck) Date: Wed, 25 Oct 2006 15:13:19 -0400 Subject: [Distutils] bdist_rpm fails due to bytecompiled modules In-Reply-To: <0D8C569F-5A77-4909-A153-24A083E47C18@mac.com> References: <20061025180500.GA27367@night.wxnet.org> <0D8C569F-5A77-4909-A153-24A083E47C18@mac.com> Message-ID: <20061025191319.GA30745@night.wxnet.org> On Wed, Oct 25, 2006 at 09:01:03PM +0200, Ronald Oussoren wrote: > > On Oct 25, 2006, at 8:05 PM, Christopher Blunck wrote: > > > > > > >the trouble here with that %install section is that the python modules > >get bytecompiled and thus turned into native code. i'd prefer for > >that not to happen, so i have written my own install.sh script and i > >provide that to my python setup.py call as follows: > > .pyc and .pyo files aren't native code, they are bytecode files that > work on all platforms. There should therefore be no need to surpress > bytecode compilation. oh i see... my mistake. :) how do you workaround the "installed (but unpackaged) file(s) found" error related to the .pyo/.pyc files? i noticed it's a python bug as well as a redhat bug. ps - the original reason i went down the rabbit hole of --optimization 0 and --no-compile was that i was trying to get around the installed but unpackaged files error. my thought was that if i could prevent the creation of the .pyc and .pyo files then i could avoid the rpmbuild error. -c -- WeatherNet Observations for station: home Temperature: 54.20F Pressure: 30.02in; Dew Point: 33.22F (45%) Wind: 64 at 3 mph Recorded: 19:04:50 10/25/06 (http://wsdl.wxnet.org/inquiry/binding.wsdl) From icon at fedoraproject.org Wed Oct 25 23:15:32 2006 From: icon at fedoraproject.org (Konstantin Ryabitsev) Date: Wed, 25 Oct 2006 17:15:32 -0400 Subject: [Distutils] bdist_rpm fails due to bytecompiled modules In-Reply-To: <20061025191319.GA30745@night.wxnet.org> References: <20061025180500.GA27367@night.wxnet.org> <0D8C569F-5A77-4909-A153-24A083E47C18@mac.com> <20061025191319.GA30745@night.wxnet.org> Message-ID: On 10/25/06, Christopher Blunck wrote: > how do you workaround the "installed (but unpackaged) file(s) found" > error related to the .pyo/.pyc files? i noticed it's a python bug as > well as a redhat bug. Not sure what you call a bug here. The creation of .pyc and .pyo files is an important stage and shouldn't be omitted when building RPMs, nor should the files be skipped (deleted and not included in %files). This way stuff like SELinux permissions get set correctly when the RPM is installed. Cheers, -- Konstantin Ryabitsev Montr?al, Qu?bec From chris at wxnet.org Wed Oct 25 23:37:59 2006 From: chris at wxnet.org (Christopher Blunck) Date: Wed, 25 Oct 2006 17:37:59 -0400 Subject: [Distutils] bdist_rpm fails due to bytecompiled modules In-Reply-To: References: <20061025180500.GA27367@night.wxnet.org> <0D8C569F-5A77-4909-A153-24A083E47C18@mac.com> <20061025191319.GA30745@night.wxnet.org> Message-ID: <20061025213759.GA2273@night.wxnet.org> On Wed, Oct 25, 2006 at 05:15:32PM -0400, Konstantin Ryabitsev wrote: > On 10/25/06, Christopher Blunck wrote: > >how do you workaround the "installed (but unpackaged) file(s) found" > >error related to the .pyo/.pyc files? i noticed it's a python bug as > >well as a redhat bug. > > Not sure what you call a bug here. The creation of .pyc and .pyo files > is an important stage and shouldn't be omitted when building RPMs, nor > should the files be skipped (deleted and not included in %files). This > way stuff like SELinux permissions get set correctly when the RPM is > installed. distutils/setuptools creates .pyc files - that's ok. here's how the problem occurs: 1.) distutils generates a .spec file with an %install that calls setup.py like below. .py and .pyc files are generated, and are included in the INSTALLED_FILES record. see below for what is called in %install: python setup.py install --single-version-externally-managed --root=$RPM_BUILD_ROOT --record=INSTALLED_FILES 2.) rpmbuild is called, and it generates a shell script that calls the command above. AFTER that command is executed, rpmbuild calls some strip commands as well as brp-python-bytecompile. brp-python-bytecompile generates .pyo files from .pyc but it does not update INSTALLED_FILES (because that is a distutils artifact that rpmbuild doesn't know about) 3.) rpmbuild calls into %install, and INSTALLED_FILES is loaded. it does not include the .pyo files that rpmbuild created. since there are unpackaged files (the .pyo files) that exist in the installation directory that are not referenced in the INSTALLED_FILES file, rpmbuild chokes with an error. step #3 is where the error occurs. it's basically an inconsistency between distutils and rpmbuild. i can run distutils to generate optimized modules (.pyo) but that doesn't apply to my scripts. as a result i still have unpackaged files... -c -- WeatherNet Observations for station: home Temperature: 51.60F Pressure: 30.04in; Dew Point: 33.02F (49%) Wind: 81 at 0 mph Recorded: 21:24:56 10/25/06 (http://wsdl.wxnet.org/inquiry/binding.wsdl) From timcera at earthlink.net Thu Oct 26 06:39:37 2006 From: timcera at earthlink.net (Tim Cera) Date: Thu, 26 Oct 2006 00:39:37 -0400 Subject: [Distutils] How to get list of package requirements from PyPI without downloading egg? Message-ID: <200610260039.37097.timcera@earthlink.net> Hello, I want the list of package requirements, which I hoped would be real simple, but have not been able to find a good solution. I could download the egg and look in the EGG_INFO directory, but is there any way to have setuptools develop the list for me? And hopefully not have to download the egg. If the above idea doesn't go anywhere - is it possible that the package requirements could be added to CheeseShop's XMLRPC? Kindest regards, Tim Cera From pierre at saiph.com Thu Oct 26 07:19:16 2006 From: pierre at saiph.com (Pierre Imbaud) Date: Thu, 26 Oct 2006 01:19:16 -0400 Subject: [Distutils] disappointed by setupTools In-Reply-To: <1161035186.5853.7.camel@nny> References: <1161035186.5853.7.camel@nny> Message-ID: Matt Good wrote: > On Mon, 2006-10-16 at 16:57 -0400, Pierre Imbaud wrote: > >>I just discovered scripts dont get installed directly (in >>/usr/local/bin), instead a wrapper in installed, that calls the >>script. So an extra file is involved, and extra code, every time the >>script is called. Makes some overhead, and it breaks a simple >>mechanism I use: my script is able to perform many actions, based on >>its name (much like cp, mv and ln are the same executable). at >>install, I need to make symlinks to the script, one per action, and >>this was automated: this is broken by the wrapping. No big deal, just >>makes things less simple. > > > Actually installing scripts with setuptools is very easy: > http://peak.telecommunity.com/DevCenter/setuptools#automatic-script-creation > > In your entry points simply put the names of the scripts you want and > the Python function that script will run. Now you don't need to do your > own symlink hack to get it all to work. Just put the functions into a > Python module and list them in setup.py. works fine, indeed. problem was I already had this set of functions, but with some general processing before functions get called (for arguments, for instance): Had to dismantle all this to adopt this new policy. > > The setuptools method also fixes problems with making scripts work in a > cross-platform manner, makes sure your scripts work correctly if someone > installs two versions of the same package on their system, and ensures > that scripts are run with the correct Python version in case it's not > the default Python executable on the system. > granted. From pierre at saiph.com Thu Oct 26 07:25:45 2006 From: pierre at saiph.com (Pierre Imbaud) Date: Thu, 26 Oct 2006 01:25:45 -0400 Subject: [Distutils] disappointed by setupTools In-Reply-To: <6a36e7290610161454pf27a691id215e13a7b135345@mail.gmail.com> References: <6a36e7290610161454pf27a691id215e13a7b135345@mail.gmail.com> Message-ID: Bob Ippolito wrote: > On 10/16/06, Pierre Imbaud wrote: > >>Im a little bit disappointed by setupTools. Not that much the tool itself, >>than the learning curve. >>Maybe the problem at hand is far from simple, and thats what makes the >>solution so intricate. >>http://peak.telecommunity.com/DevCenter/setuptools: this document is >>100 pages long! (by page I mean: full screen). >>easyInstall is 40 pages. >>I spent about 3 days to figure out what goes where. And I realize Im >>not done. >>Yet the documentation is remarkable, great presentation, text is >>precise and clear, when U read it. Makes U feel guilty U dont find >>what U lookin for, but U dont find it. > > > The current documentation is a reference, not a tutorial. It's long > because it's exhaustive. What's needed is different documentation for > new users. indeed > > >>I just discovered scripts dont get installed directly (in >>/usr/local/bin), instead a wrapper in installed, that calls the >>script. So an extra file is involved, and extra code, every time the >>script is called. Makes some overhead, and it breaks a simple >>mechanism I use: my script is able to perform many actions, based on >>its name (much like cp, mv and ln are the same executable). at >>install, I need to make symlinks to the script, one per action, and >>this was automated: this is broken by the wrapping. No big deal, just >>makes things less simple. >>Good tools have 2 qualities: >>- When U need simple things, its easy to use. >>- When U need more sophisticated things, U dont need to give up the >> tool for a more powerful one: just have to use extra features. >>Python is a wonderful illustration of this. >>While setupTools obviously provides the latter, I consider it fails on >>the former. >>I decided to use setupTools after reading, and failing to understand, >>distutils. setupTools seemed both more powerful and simpler. >>Probably this choice will save me some trouble later. Im not sure it >>helped so far. > > > It doesn't work because you're doing it the wrong way. If you want to > see what a command was invoked as, use sys.argv[0]... that's what it's > for. Thats what I did, thats the obvious way. But with this wrapper policy, sys.argv[0] is the wrapper. to get to the module name, I needed __file__! > > Also, you don't *have* to use entry points to make scripts (though > it's generally better to)... the scripts kwarg to setup works the same > way that it does for distutils. > > -bob > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > Thanks both of U for your answers. The tool is handy indeed... once U climbed the somewhat steep learning curve. From pje at telecommunity.com Thu Oct 26 07:51:39 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 26 Oct 2006 01:51:39 -0400 Subject: [Distutils] How to get list of package requirements from PyPI without downloading egg? In-Reply-To: <200610260039.37097.timcera@earthlink.net> Message-ID: <5.1.1.6.0.20061026014748.02c37a30@sparrow.telecommunity.com> At 12:39 AM 10/26/2006 -0400, Tim Cera wrote: >Hello, > >I want the list of package requirements, which I hoped would be real >simple, but have not been able to find a good solution. > >I could download the egg and look in the EGG_INFO directory, but is >there any way to have setuptools develop the list for me? Use the pkg_resources module, specifically the requires() method of Distribution objects: http://peak.telecommunity.com/DevCenter/PkgResources#distribution-methods Here are some ways to obtain Distribution objects: http://peak.telecommunity.com/DevCenter/PkgResources#getting-or-creating-distributions > And >hopefully not have to download the egg. No, sorry, you have to have the egg to extract its requirements. >If the above idea doesn't go anywhere - is it possible that the package >requirements could be added to CheeseShop's XMLRPC? Requirements can be Python version and platform-specific, so I'm afraid this can't be reasonably captured in a Cheeseshop entry at the present time. From pje at telecommunity.com Thu Oct 26 07:53:16 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 26 Oct 2006 01:53:16 -0400 Subject: [Distutils] disappointed by setupTools In-Reply-To: References: <6a36e7290610161454pf27a691id215e13a7b135345@mail.gmail.com> <6a36e7290610161454pf27a691id215e13a7b135345@mail.gmail.com> Message-ID: <5.1.1.6.0.20061026015225.04baca28@sparrow.telecommunity.com> At 01:25 AM 10/26/2006 -0400, Pierre Imbaud wrote: >Thats what I did, thats the obvious way. But with this wrapper policy, >sys.argv[0] is the wrapper. to get to the module name, I needed __file__! Perhaps you mean __name__? From jeremy.kloth at 4suite.org Thu Oct 26 08:05:37 2006 From: jeremy.kloth at 4suite.org (Jeremy Kloth) Date: Thu, 26 Oct 2006 00:05:37 -0600 Subject: [Distutils] How to get list of package requirements from PyPI without downloading egg? In-Reply-To: <5.1.1.6.0.20061026014748.02c37a30@sparrow.telecommunity.com> References: <5.1.1.6.0.20061026014748.02c37a30@sparrow.telecommunity.com> Message-ID: <200610260005.37824.jeremy.kloth@4suite.org> On Wednesday 25 October 2006 11:51 pm, Phillip J. Eby wrote: > At 12:39 AM 10/26/2006 -0400, Tim Cera wrote: > >If the above idea doesn't go anywhere - is it possible that the package > >requirements could be added to CheeseShop's XMLRPC? > > Requirements can be Python version and platform-specific, so I'm afraid > this can't be reasonably captured in a Cheeseshop entry at the present > time. Except Cheeseshop already supports the PEP 314 metadeta fields Requires, Provides and Obsoletes. They are available to all packages built with Python 2.5 distutils. That said, the community still needs to agree on what the names of the requirements are. The Cheeseshop restrictions match package names and that is what 4Suite's PackageManager uses for its lookup routines. It solves the Python version problem by defining the PEP 345 metadata field Requires-Python. -- Jeremy Kloth http://4suite.org/ From pje at telecommunity.com Thu Oct 26 08:21:22 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 26 Oct 2006 02:21:22 -0400 Subject: [Distutils] How to get list of package requirements from PyPI without downloading egg? In-Reply-To: <200610260005.37824.jeremy.kloth@4suite.org> References: <5.1.1.6.0.20061026014748.02c37a30@sparrow.telecommunity.com> <5.1.1.6.0.20061026014748.02c37a30@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20061026020814.0277ea18@sparrow.telecommunity.com> At 12:05 AM 10/26/2006 -0600, Jeremy Kloth wrote: >On Wednesday 25 October 2006 11:51 pm, Phillip J. Eby wrote: > > At 12:39 AM 10/26/2006 -0400, Tim Cera wrote: > > >If the above idea doesn't go anywhere - is it possible that the package > > >requirements could be added to CheeseShop's XMLRPC? > > > > Requirements can be Python version and platform-specific, so I'm afraid > > this can't be reasonably captured in a Cheeseshop entry at the present > > time. > >Except Cheeseshop already supports the PEP 314 metadeta fields Requires, >Provides and Obsoletes. They are available to all packages built with Python >2.5 distutils. Indeed it does... but that information is both unused and useless at the moment. >That said, the community still needs to agree on what the names of the >requirements are. Yep - like I said, it's both unused and useless. :) >It solves the Python version problem by defining the PEP 345 metadata field >Requires-Python. No, it doesn't, actually. My statement was that the requirements *themselves* can be version and platform specific. PEP 314 only allows for one set of requirements for a package, not one set of requirements per version and platform. In any case, saying that we have a solution and that we just have to decide what it means, reminds me of a passage from the Douglas Adams novel, "Dirk Gently's Holistic Detective Agency", wherein Dirk makes a cryptic scrawl on a piece of paper and declares that it is the solution to the mystery! They need only determine what language he wrote it in, and translate it back to English... :) Personally, I think PEP 314 should be rejected, as this issue is only one of its flaws. I'm surprised to see it is now marked "final", given the near total lack of discussion on it, except for me periodically posting about its flaws (since 2005, no less). None of the new fields except Download-URL are actually useful. PEP 314 delenda est. From jeremy.kloth at 4suite.org Thu Oct 26 08:53:26 2006 From: jeremy.kloth at 4suite.org (Jeremy Kloth) Date: Thu, 26 Oct 2006 00:53:26 -0600 Subject: [Distutils] How to get list of package requirements from PyPI without downloading egg? In-Reply-To: <5.1.1.6.0.20061026020814.0277ea18@sparrow.telecommunity.com> References: <5.1.1.6.0.20061026014748.02c37a30@sparrow.telecommunity.com> <5.1.1.6.0.20061026020814.0277ea18@sparrow.telecommunity.com> Message-ID: <200610260053.26353.jeremy.kloth@4suite.org> On Thursday 26 October 2006 12:21 am, Phillip J. Eby wrote: > At 12:05 AM 10/26/2006 -0600, Jeremy Kloth wrote: > >Except Cheeseshop already supports the PEP 314 metadeta fields Requires, > >Provides and Obsoletes. They are available to all packages built with > > Python 2.5 distutils. > > Indeed it does... but that information is both unused and useless at the > moment. Could it be that it is unused because Pyhon 2.5 was just released? That is when these fields made it into Distutils. > >That said, the community still needs to agree on what the names of the > >requirements are. > > Yep - like I said, it's both unused and useless. :) > > >It solves the Python version problem by defining the PEP 345 metadata > > field Requires-Python. > > No, it doesn't, actually. My statement was that the requirements > *themselves* can be version and platform specific. PEP 314 only allows for > one set of requirements for a package, not one set of requirements per > version and platform. So how do you specify an exact platform and Python version with setuptools' install_requires? They are, IMO, used at the same level. > Personally, I think PEP 314 should be rejected, as this issue is only one > of its flaws. I'm surprised to see it is now marked "final", given the > near total lack of discussion on it, except for me periodically posting > about its flaws (since 2005, no less). None of the new fields except > Download-URL are actually useful. It is a little late now. Those fields are part of Python 2.5 so they are going to be around for at least 3 more years (if the 2.3 to 2.5 timeline is any indicator). So how about doing something to actually *improve* the fields besides just complaining? Going forward with (probable) modifications to the fields' allowed values, it should be fairly easy to expand the allowed values as the need arises since the current values are so restrictive. > PEP 314 delenda est. Now there is a postitive attitude... -- Jeremy Kloth http://4suite.org/ From pje at telecommunity.com Thu Oct 26 17:21:43 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 26 Oct 2006 11:21:43 -0400 Subject: [Distutils] How to get list of package requirements from PyPI without downloading egg? In-Reply-To: <200610260053.26353.jeremy.kloth@4suite.org> References: <5.1.1.6.0.20061026020814.0277ea18@sparrow.telecommunity.com> <5.1.1.6.0.20061026014748.02c37a30@sparrow.telecommunity.com> <5.1.1.6.0.20061026020814.0277ea18@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20061026110000.0272bd30@sparrow.telecommunity.com> At 12:53 AM 10/26/2006 -0600, Jeremy Kloth wrote: >On Thursday 26 October 2006 12:21 am, Phillip J. Eby wrote: > > At 12:05 AM 10/26/2006 -0600, Jeremy Kloth wrote: > > >Except Cheeseshop already supports the PEP 314 metadeta fields Requires, > > >Provides and Obsoletes. They are available to all packages built with > > > Python 2.5 distutils. > > > > Indeed it does... but that information is both unused and useless at the > > moment. > >Could it be that it is unused because Pyhon 2.5 was just released? That is >when these fields made it into Distutils. That could be why it's unused; but not why it's useless. No agreement on semantics simply means that the odds are astronomically against anyone putting in data that can actually be used by someone else. You would *first* need to create a tool that could do something useful with the information, and second, you'd need it to be something that actually *tested* the usefulness -- meaning you'd need a distutils extension, or else you'd have to wait until 2.6 to add new features to distutils itself. (If the data can't be tested by the person putting it in their setup() script, then it will be wrong, and therefore useless to others for any sort of automated dependency resolution.) >So how do you specify an exact platform and Python version with setuptools' >install_requires? They are, IMO, used at the same level. install_requires is set by a script, so the script can set it based on the Python version and platform the script is running on. Thus, eggs built for different platforms or Python versions can have different requirements. > > Personally, I think PEP 314 should be rejected, as this issue is only one > > of its flaws. I'm surprised to see it is now marked "final", given the > > near total lack of discussion on it, except for me periodically posting > > about its flaws (since 2005, no less). None of the new fields except > > Download-URL are actually useful. > >It is a little late now. Those fields are part of Python 2.5 so they are >going to be around for at least 3 more years (if the 2.3 to 2.5 timeline is >any indicator). So how about doing something to actually *improve* the fields >besides just complaining? If there were a *way* to improve them, I'd have suggested it a long time ago. >Going forward with (probable) modifications to the fields' allowed values, it >should be fairly easy to expand the allowed values as the need arises since >the current values are so restrictive. Until somebody has some idea of 1) what the semantics are, and 2) what the use cases for those semantics are, the fields aren't of any value whatsoever. This isn't something that's going to be improved by fiddling with their syntax, so long as the semantics remain undefined. > > PEP 314 delenda est. > >Now there is a postitive attitude... Well, I'm pretty positive that it delenda est. :) In truth, however, it's just a lame duck that won't fly, doomed to irrelevance in the bigger scheme of things. It would take a heroic effort to make anything useful out of it, and there's no incentive for anybody to make that effort. So, from my POV, the only useful forward action is to put it out of its misery, since the only other thing that's ever going to happen with it is talk, and maybe a few people putting some useless and incompatible things in there. You need some kind of community that wants to specify dependencies, but *doesn't* want anybody to be able to automatically install them! Or else, that develops its own set of tools for automatic installation... that only works with packages whose users followed that new tool's conventions. In effect, you need a community large enough to create its own mini-PyPI... and that isn't already using setuptools dependencies to do the same thing. Or, you need to have it use the metadata in a style similar to setuptools, in which case you're limited by PEP 314's crazy syntax restrictions, that aren't compatible with existing PyPI distribution names. About the only thing that could possibly outweigh all that is a BDFL proclamation in favor of some particular semantics... which nobody has proposed or agrees upon as yet. From jeremy.kloth at 4suite.org Thu Oct 26 18:20:41 2006 From: jeremy.kloth at 4suite.org (Jeremy Kloth) Date: Thu, 26 Oct 2006 10:20:41 -0600 Subject: [Distutils] How to get list of package requirements from PyPI without downloading egg? In-Reply-To: <5.1.1.6.0.20061026110000.0272bd30@sparrow.telecommunity.com> References: <5.1.1.6.0.20061026020814.0277ea18@sparrow.telecommunity.com> <5.1.1.6.0.20061026110000.0272bd30@sparrow.telecommunity.com> Message-ID: <200610261020.42624.jeremy.kloth@4suite.org> On Thursday 26 October 2006 9:21 am, Phillip J. Eby wrote: > At 12:53 AM 10/26/2006 -0600, Jeremy Kloth wrote: > >On Thursday 26 October 2006 12:21 am, Phillip J. Eby wrote: > > No agreement on semantics simply means that the odds are astronomically > against anyone putting in data that can actually be used by someone else. As you like to point out so often, I do not actually recall anyone has ever brought up the semantics of the fields except yourself whom obviously is 110% against them to begin with. > You would *first* need to create a tool that could do something useful with > the information, Done. 4Suite's PackageManager. > and second, you'd need it to be something that actually *tested* the > usefulness -- meaning you'd need a distutils extension, Done. See above. > or else you'd have to wait until 2.6 to add new features to distutils > itself. Hmm, interesting idea. It so happens that I've been using/extending distutils since its introduction and am *very* comfortable with its code base. I believe that basically an offer to take maintenanceship of it was extended on python-dev, this may be something that I will look into. There wouldn't need to be wait if distutils itself starting making separate releases again. > install_requires is set by a script, so the script can set it based on the > Python version and platform the script is running on. Thus, eggs built for > different platforms or Python versions can have different requirements. And you're telling me that the `requires` argument isn't set by a script? A platform/version specific dependency issue can only come into play if a particular distribution is begin scanned for dependencies out-of-band, which is what the OP was talking about, but that is not what is at issue here. > Until somebody has some idea of 1) what the semantics are, and 2) what the > use cases for those semantics are, the fields aren't of any value > whatsoever. This isn't something that's going to be improved by fiddling > with their syntax, so long as the semantics remain undefined. Aparently you haven't used a recent Linux distribution? Both apt or rpm have the semantics clearly defined and that is the first thing that came to my mind when I came across those fields. Are you sure that you are not just against things that are not setuptools? -- Jeremy Kloth http://4suite.org/ From pje at telecommunity.com Thu Oct 26 21:14:40 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 26 Oct 2006 15:14:40 -0400 Subject: [Distutils] How to get list of package requirements from PyPI without downloading egg? In-Reply-To: <200610261020.42624.jeremy.kloth@4suite.org> References: <5.1.1.6.0.20061026110000.0272bd30@sparrow.telecommunity.com> <5.1.1.6.0.20061026020814.0277ea18@sparrow.telecommunity.com> <5.1.1.6.0.20061026110000.0272bd30@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20061026131434.029d95b0@sparrow.telecommunity.com> At 10:20 AM 10/26/2006 -0600, Jeremy Kloth wrote: >On Thursday 26 October 2006 9:21 am, Phillip J. Eby wrote: > > At 12:53 AM 10/26/2006 -0600, Jeremy Kloth wrote: > > >On Thursday 26 October 2006 12:21 am, Phillip J. Eby wrote: > > > > No agreement on semantics simply means that the odds are astronomically > > against anyone putting in data that can actually be used by someone else. > >As you like to point out so often, I do not actually recall anyone has ever >brought up the semantics of the fields except yourself whom obviously is 110% >against them to begin with. > > > You would *first* need to create a tool that could do something useful > with > > the information, > >Done. 4Suite's PackageManager. > > > and second, you'd need it to be something that actually *tested* the > > usefulness -- meaning you'd need a distutils extension, > >Done. See above. Sounds great, then. What's the problem? (Btw, do you have a documentation link? Google doesn't turn up anything except API docs that say nothing about the PEP 314 stuff.) > > install_requires is set by a script, so the script can set it based on the > > Python version and platform the script is running on. Thus, eggs built for > > different platforms or Python versions can have different requirements. > >And you're telling me that the `requires` argument isn't set by a script? No, I'm saying that the Cheeseshop only holds one set of "requires" values. >A platform/version specific dependency issue can only come into play if a >particular distribution is begin scanned for dependencies out-of-band, which >is what the OP was talking about, but that is not what is at issue here. I was assuming we were talking about the OP's request, so that explains the disconnect. > > Until somebody has some idea of 1) what the semantics are, and 2) what the > > use cases for those semantics are, the fields aren't of any value > > whatsoever. This isn't something that's going to be improved by fiddling > > with their syntax, so long as the semantics remain undefined. > >Aparently you haven't used a recent Linux distribution? Both apt or rpm have >the semantics clearly defined and that is the first thing that came to my >mind when I came across those fields. Well, the PEP explicitly says there is *no* semantics defined, so I chose to believe the PEP author. :) (Yes, I'm familiar with RPM-style requirements.) > Are you sure that you are not just against things that are not setuptools? Quite sure. PEP 314 was the first place I looked for an opportunity to implement setuptools' requirements. However, I found it to be inadequate for my goals, which included backward-compatibility with existing distutils-based packages, and support for simple, statically-published package indexes and links. In essence, the "provides" portion of the PEP makes it impossible (AFAICT) to provide these features. In my mind, supporting discovery of non-setuptools packages and allowing distributed package publishing was more important than mimicking RPM. That's why I'm saying that getting PEP 314-based stuff out into the wild requires either ignoring PEP 314's syntax restrictions (and the provides/obsoletes fields along with them), or a break with both backward compatibility and distributed publishing. So, my opinion isn't really a matter of objecting to a "competing" system (which would be silly). It's more a prediction that PEP 314 isn't really going to go anywhere, because it imposes the cost of adoption on the wrong people, without providing them any countervailing benefits. So it doesn't take any great leap of intuition to see that it will go nowhere unless something changes. That having been said, if I'm wrong and it *does* succeed in going somewhere (e.g. the BDFL blesses some set of semantics), then I'll certainly look at what needs to be done to add support for it to setuptools. But at this point, it just looks to me like one of the perennial "we should do something about X" ideas that doesn't go anywhere because there isn't enough critical mass for adoption or anybody who knows how to solve the technical problems. From jeremy.kloth at 4suite.org Thu Oct 26 21:36:49 2006 From: jeremy.kloth at 4suite.org (Jeremy Kloth) Date: Thu, 26 Oct 2006 13:36:49 -0600 Subject: [Distutils] How to get list of package requirements from PyPI without downloading egg? In-Reply-To: <5.1.1.6.0.20061026131434.029d95b0@sparrow.telecommunity.com> References: <5.1.1.6.0.20061026110000.0272bd30@sparrow.telecommunity.com> <5.1.1.6.0.20061026131434.029d95b0@sparrow.telecommunity.com> Message-ID: <200610261336.49991.jeremy.kloth@4suite.org> On Thursday 26 October 2006 1:14 pm, Phillip J. Eby wrote: > (Btw, do you have a documentation > link? Google doesn't turn up anything except API docs that say nothing > about the PEP 314 stuff.) http://skew.org/~mike/4suite-docs/html/modules/Ft.Lib.DistExt.Dist.html#Dist or (as always, "Read the source, Luke"): http://cvs.4suite.org/viewcvs/4Suite/Ft/Lib/DistExt/Dist.py?view=markup The entire body of Ft.Lib.DistExt contains many things that we as developers found quite useful. Even Guido himself has shown interest in some of our extensions. It is just recently that I'm of the mind to unleash DistExt from its 4Suite-XML binds. -- Jeremy Kloth http://4suite.org/ From pje at telecommunity.com Thu Oct 26 21:45:55 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 26 Oct 2006 15:45:55 -0400 Subject: [Distutils] How to get list of package requirements from PyPI without downloading egg? In-Reply-To: <200610261336.49991.jeremy.kloth@4suite.org> References: <5.1.1.6.0.20061026131434.029d95b0@sparrow.telecommunity.com> <5.1.1.6.0.20061026110000.0272bd30@sparrow.telecommunity.com> <5.1.1.6.0.20061026131434.029d95b0@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20061026154138.02715bd0@sparrow.telecommunity.com> At 01:36 PM 10/26/2006 -0600, Jeremy Kloth wrote: >On Thursday 26 October 2006 1:14 pm, Phillip J. Eby wrote: > > (Btw, do you have a documentation > > link? Google doesn't turn up anything except API docs that say nothing > > about the PEP 314 stuff.) > >http://skew.org/~mike/4suite-docs/html/modules/Ft.Lib.DistExt.Dist.html#Dist > >or (as always, "Read the source, Luke"): > >http://cvs.4suite.org/viewcvs/4Suite/Ft/Lib/DistExt/Dist.py?view=markup I'm still lost. What are you *doing* with the requires, provides, and obsoletes fields, other than storing or outputting them? There's nothing in the doc or the source linked above that actually *does* anything with the data that I can see. How do you use this to find and install dependencies, for example? From jeremy.kloth at 4suite.org Thu Oct 26 21:58:30 2006 From: jeremy.kloth at 4suite.org (Jeremy Kloth) Date: Thu, 26 Oct 2006 13:58:30 -0600 Subject: [Distutils] How to get list of package requirements from PyPI without downloading egg? In-Reply-To: <5.1.1.6.0.20061026154138.02715bd0@sparrow.telecommunity.com> References: <5.1.1.6.0.20061026131434.029d95b0@sparrow.telecommunity.com> <5.1.1.6.0.20061026154138.02715bd0@sparrow.telecommunity.com> Message-ID: <200610261358.31034.jeremy.kloth@4suite.org> On Thursday 26 October 2006 1:45 pm, Phillip J. Eby wrote: > At 01:36 PM 10/26/2006 -0600, Jeremy Kloth wrote: > >On Thursday 26 October 2006 1:14 pm, Phillip J. Eby wrote: > > > (Btw, do you have a documentation > > > link? Google doesn't turn up anything except API docs that say nothing > > > about the PEP 314 stuff.) > > > >http://skew.org/~mike/4suite-docs/html/modules/Ft.Lib.DistExt.Dist.html#Di > >st > > > >or (as always, "Read the source, Luke"): > > > >http://cvs.4suite.org/viewcvs/4Suite/Ft/Lib/DistExt/Dist.py?view=markup > > I'm still lost. What are you *doing* with the requires, provides, and > obsoletes fields, other than storing or outputting them? Well, you asked where they were mentioned, not used ;) > There's nothing > in the doc or the source linked above that actually *does* anything with > the data that I can see. How do you use this to find and install > dependencies, for example? The consumer of those fields is actually the PackageManager class: http://cvs.4suite.org/viewcvs/4Suite/Ft/Lib/DistExt/PackageManager.py?view=markup -- Jeremy Kloth http://4suite.org/ From icon at fedoraproject.org Thu Oct 26 22:23:01 2006 From: icon at fedoraproject.org (Konstantin Ryabitsev) Date: Thu, 26 Oct 2006 16:23:01 -0400 Subject: [Distutils] bdist_rpm fails due to bytecompiled modules In-Reply-To: <20061025213759.GA2273@night.wxnet.org> References: <20061025180500.GA27367@night.wxnet.org> <0D8C569F-5A77-4909-A153-24A083E47C18@mac.com> <20061025191319.GA30745@night.wxnet.org> <20061025213759.GA2273@night.wxnet.org> Message-ID: On 10/25/06, Christopher Blunck wrote: > > Not sure what you call a bug here. The creation of .pyc and .pyo files > > is an important stage and shouldn't be omitted when building RPMs, nor > > should the files be skipped (deleted and not included in %files). This > > way stuff like SELinux permissions get set correctly when the RPM is > > installed. > > distutils/setuptools creates .pyc files - that's ok. > > here's how the problem occurs: > > 1.) distutils generates a .spec file with an %install that calls > setup.py like below. .py and .pyc files are generated, and are > included in the INSTALLED_FILES record. see below for what is > called in %install: > > python setup.py install --single-version-externally-managed --root=$RPM_BUILD_ROOT --record=INSTALLED_FILES > > > 2.) rpmbuild is called, and it generates a shell script that calls the > command above. AFTER that command is executed, rpmbuild calls some > strip commands as well as brp-python-bytecompile. > brp-python-bytecompile generates .pyo files from .pyc but it does not > update INSTALLED_FILES (because that is a distutils artifact that > rpmbuild doesn't know about) > > 3.) rpmbuild calls into %install, and INSTALLED_FILES is loaded. it > does not include the .pyo files that rpmbuild created. since > there are unpackaged files (the .pyo files) that exist in the > installation directory that are not referenced in the > INSTALLED_FILES file, rpmbuild chokes with an error. > > > step #3 is where the error occurs. it's basically an inconsistency > between distutils and rpmbuild. i can run distutils to generate > optimized modules (.pyo) but that doesn't apply to my scripts. as a > result i still have unpackaged files... Yes, it looks like distutils is not anticipating the new behaviour from rpmbuild (both .pyc and .pyo will be compiled by brp-python-bytecompile). It's probably something that should be fixed when generating the %files section. Regards, -- Konstantin Ryabitsev Montr?al, Qu?bec From pje at telecommunity.com Thu Oct 26 22:41:40 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 26 Oct 2006 16:41:40 -0400 Subject: [Distutils] How to get list of package requirements from PyPI without downloading egg? In-Reply-To: <200610261358.31034.jeremy.kloth@4suite.org> References: <5.1.1.6.0.20061026154138.02715bd0@sparrow.telecommunity.com> <5.1.1.6.0.20061026131434.029d95b0@sparrow.telecommunity.com> <5.1.1.6.0.20061026154138.02715bd0@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20061026164030.02716060@sparrow.telecommunity.com> At 01:58 PM 10/26/2006 -0600, Jeremy Kloth wrote: >The consumer of those fields is actually the PackageManager class: > >http://cvs.4suite.org/viewcvs/4Suite/Ft/Lib/DistExt/PackageManager.py?view=markup Yeah, I'm still not getting what it *does* with this information. What's it *for*? From gwk.rko at googlemail.com Thu Oct 26 22:44:47 2006 From: gwk.rko at googlemail.com (Georg) Date: Thu, 26 Oct 2006 22:44:47 +0200 Subject: [Distutils] permission / umask on install? Message-ID: <5b9e71800610261344pc71a260t321056ed09b3f043@mail.gmail.com> Hi, I was somewhat surprised to find that distutils on install does honor the current umask. If run as root and root's umask is 027 (a common case nowadays), stuff will be installed as non world readable / executable. Compare this to install(1)'s behavior on Unix. Shouldn't the default be to DTRT on Unix? Or shouldn't there be at least an option to specify the target permissions for data and executable files? -- Regards, Georg. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20061026/7ea524bd/attachment.html From jeremy.kloth at 4suite.org Thu Oct 26 23:04:22 2006 From: jeremy.kloth at 4suite.org (Jeremy Kloth) Date: Thu, 26 Oct 2006 15:04:22 -0600 Subject: [Distutils] How to get list of package requirements from PyPI without downloading egg? In-Reply-To: <5.1.1.6.0.20061026164030.02716060@sparrow.telecommunity.com> References: <5.1.1.6.0.20061026154138.02715bd0@sparrow.telecommunity.com> <5.1.1.6.0.20061026164030.02716060@sparrow.telecommunity.com> Message-ID: <200610261504.22600.jeremy.kloth@4suite.org> On Thursday 26 October 2006 2:41 pm, Phillip J. Eby wrote: > At 01:58 PM 10/26/2006 -0600, Jeremy Kloth wrote: > >The consumer of those fields is actually the PackageManager class: > > > >http://cvs.4suite.org/viewcvs/4Suite/Ft/Lib/DistExt/PackageManager.py?view > >=markup > > Yeah, I'm still not getting what it *does* with this information. What's > it *for*? Since there is no current way to get the requires/provides/obsoletes from PyPI RPC, they are used to determine the sub-package installation order and to verify that the requirements are installed if not already provided by another sub-package. PackageManager doesn't do any downloading as it is pointless until those fields become available through RPC. And no, I have no desire to do screen scraping to get their values. -- Jeremy Kloth http://4suite.org/ From pje at telecommunity.com Fri Oct 27 00:13:01 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 26 Oct 2006 18:13:01 -0400 Subject: [Distutils] How to get list of package requirements from PyPI without downloading egg? In-Reply-To: <200610261504.22600.jeremy.kloth@4suite.org> References: <5.1.1.6.0.20061026164030.02716060@sparrow.telecommunity.com> <5.1.1.6.0.20061026154138.02715bd0@sparrow.telecommunity.com> <5.1.1.6.0.20061026164030.02716060@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20061026180838.026dd938@sparrow.telecommunity.com> At 03:04 PM 10/26/2006 -0600, Jeremy Kloth wrote: >On Thursday 26 October 2006 2:41 pm, Phillip J. Eby wrote: > > At 01:58 PM 10/26/2006 -0600, Jeremy Kloth wrote: > > >The consumer of those fields is actually the PackageManager class: > > > > > >http://cvs.4suite.org/viewcvs/4Suite/Ft/Lib/DistExt/PackageManager.py?view > > >=markup > > > > Yeah, I'm still not getting what it *does* with this information. What's > > it *for*? > >Since there is no current way to get the requires/provides/obsoletes from >PyPI >RPC, they are used to determine the sub-package installation order and to >verify that the requirements are installed if not already provided by another >sub-package. What's a sub-package? >PackageManager doesn't do any downloading as it is pointless until those >fields become available through RPC. Okay, I thought you said this was a tool that did something useful with the PEP 314 data. So I'm confused now. This sub-package thing, if I'm getting the gist correctly, only works with stuff you're actually *bundling*, so what difference does PyPI make? From john at thegreens.co.uk Fri Oct 27 21:56:56 2006 From: john at thegreens.co.uk (John Green) Date: Fri, 27 Oct 2006 20:56:56 +0100 Subject: [Distutils] Import module failure with an egg(sometimes) Message-ID: <20061027195656.GB2282@localhost.localdomain> Would appreciate some help with where to start trying to find out what's going on here. We're using eggs as the format for extensions to our app and I'm having the odd bit of strangeness on one machine. I have another with identical software setup which behaves as expected. Here is a snippet of our code: pkg_resources.working_set.add_entry(fullName) dist_generator = pkg_resources.find_distributions(fullName) for dist in dist_generator: try: extension_class = dist.load_entry_point("jokosher.extensions", "extension") extension = extension_class() print("...done.") except Exception, e : print("...failed.") print(str(e)) and usually it works. Now because can run with an experimental version of the gstreamer libraries we can run a script before we run our program which sets various ENV variables (including PYTHONPATH) to point us to the development libraries instead of the stock ones. If I run with the stock setup everything is OK. On one machine (and one machine only) if I run the setup script in order to pick up the experimental libraries then I get ImportError. This is a section of the output by running the program with 'python -vv'. First - with the stock libs (no pre-run script): importing extension... AddInstrumentType-0.1-py2.4.egg # zipimport: found 10 names in /home/john/.jokosher/extensions/AddInstrumentType-0.1-py2.4.egg # zipimport: zlib available # trying /home/john/code/jokosher/trunk/Jokosher/AddInstrumentType.so # trying /home/john/code/jokosher/trunk/Jokosher/AddInstrumentTypemodule.so # trying /home/john/code/jokosher/trunk/Jokosher/AddInstrumentType.py # trying /home/john/code/jokosher/trunk/Jokosher/AddInstrumentType.pyc # trying /usr/lib/python2.4/site-packages/gst-0.10/AddInstrumentType.so # trying /usr/lib/python2.4/site-packages/gst-0.10/AddInstrumentTypemodule.so # trying /usr/lib/python2.4/site-packages/gst-0.10/AddInstrumentType.py # trying /usr/lib/python2.4/site-packages/gst-0.10/AddInstrumentType.pyc # trying /home/john/code/jokosher/trunk/Jokosher/AddInstrumentType.so # trying /home/john/code/jokosher/trunk/Jokosher/AddInstrumentTypemodule.so # trying /home/john/code/jokosher/trunk/Jokosher/AddInstrumentType.py # trying /home/john/code/jokosher/trunk/Jokosher/AddInstrumentType.pyc # trying /usr/lib/python24.zip/AddInstrumentType.so ... ... ... # trying /home/john/code/jokosher/trunk/AddInstrumentType.so # trying /home/john/code/jokosher/trunk/AddInstrumentTypemodule.so # trying /home/john/code/jokosher/trunk/AddInstrumentType.py # trying /home/john/code/jokosher/trunk/AddInstrumentType.pyc # trying /home/john/.jokosher/extensions/AddInstrumentType-0.1-py2.4.egg/AddInstrumentType/__init __.pyc # trying /home/john/.jokosher/extensions/AddInstrumentType-0.1-py2.4.egg/AddInstrumentType/AddIns trumentType/__init__.pyc # trying /home/john/.jokosher/extensions/AddInstrumentType-0.1-py2.4.egg/AddInstrumentType/AddIns trumentType/__init__.pyo # trying /home/john/.jokosher/extensions/AddInstrumentType-0.1-py2.4.egg/AddInstrumentType/AddIns trumentType/__init__.py # trying /home/john/.jokosher/extensions/AddInstrumentType-0.1-py2.4.egg/AddInstrumentType/AddIns trumentType.pyc # trying /home/john/.jokosher/extensions/AddInstrumentType-0.1-py2.4.egg/AddInstrumentType/Jokosh er.so # trying /home/john/.jokosher/extensions/AddInstrumentType-0.1-py2.4.egg/AddInstrumentType/Jokosh ermodule.so # trying /home/john/.jokosher/extensions/AddInstrumentType-0.1-py2.4.egg/AddInstrumentType/Jokosh er.py # trying /home/john/.jokosher/extensions/AddInstrumentType-0.1-py2.4.egg/AddInstrumentType/Jokosh er.pyc # trying /home/john/.jokosher/extensions/AddInstrumentType-0.1-py2.4.egg/AddInstrumentType/gtk.so # trying /home/john/.jokosher/extensions/AddInstrumentType-0.1-py2.4.egg/AddInstrumentType/gtkmod ule.so # trying /home/john/.jokosher/extensions/AddInstrumentType-0.1-py2.4.egg/AddInstrumentType/gtk.py # trying /home/john/.jokosher/extensions/AddInstrumentType-0.1-py2.4.egg/AddInstrumentType/gtk.py c # trying /home/john/.jokosher/extensions/AddInstrumentType-0.1-py2.4.egg/AddInstrumentType/os.so # trying /home/john/.jokosher/extensions/AddInstrumentType-0.1-py2.4.egg/AddInstrumentType/osmodu le.so # trying /home/john/.jokosher/extensions/AddInstrumentType-0.1-py2.4.egg/AddInstrumentType/os.py # trying /home/john/.jokosher/extensions/AddInstrumentType-0.1-py2.4.egg/AddInstrumentType/os.pyc # trying /home/john/.jokosher/extensions/AddInstrumentType-0.1-py2.4.egg/AddInstrumentType/pkg_re sources.so # trying /home/john/.jokosher/extensions/AddInstrumentType-0.1-py2.4.egg/AddInstrumentType/pkg_re sourcesmodule.so # trying /home/john/.jokosher/extensions/AddInstrumentType-0.1-py2.4.egg/AddInstrumentType/pkg_re sources.py # trying /home/john/.jokosher/extensions/AddInstrumentType-0.1-py2.4.egg/AddInstrumentType/pkg_re sources.pyc import AddInstrumentType.AddInstrumentType # loaded from Zip /home/john/.jokosher/extensions/AddI nstrumentType-0.1-py2.4.egg/AddInstrumentType/AddInstrumentType.pyc import AddInstrumentType # loaded from Zip /home/john/.jokosher/extensions/AddInstrumentType-0.1- py2.4.egg/AddInstrumentType/__init__.pyc ...done. No problems there. But after running the env-altering script all we get out is: importing extension... AddInstrumentType-0.1-py2.4.egg # zipimport: found 10 names in /home/john/.jokosher/extensions/AddInstrumentType-0.1-py2.4.egg # zipimport: zlib available ...failed. No module named AddInstrumentType It doesn't even seem to begin to look for the module. As I said this only happens on one machine and I know that it used to work on that one too. If you could give us a clue on what to look for next then that would be appreciated. Thanks -- John Green From pje at telecommunity.com Fri Oct 27 23:21:26 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 27 Oct 2006 17:21:26 -0400 Subject: [Distutils] Import module failure with an egg(sometimes) In-Reply-To: <20061027195656.GB2282@localhost.localdomain> Message-ID: <5.1.1.6.0.20061027171913.02717fa8@sparrow.telecommunity.com> At 08:56 PM 10/27/2006 +0100, John Green wrote: >It doesn't even seem to begin to look for the module. As I said this only >happens on one machine and I know that it used to work on that one too. If >you could give us a clue on what to look for next then that would be >appreciated. I'd suggest investigating the environment variables to see which one(s) are breaking things. Also, double check your paths to make sure you don't have another AddInstrumentType directory floating around, since that might be confusing things. The other thing would be to compare environment variables and directory contents with the machine that works. From mike_mp at zzzcomputing.com Mon Oct 30 20:43:32 2006 From: mike_mp at zzzcomputing.com (Michael Bayer) Date: Mon, 30 Oct 2006 14:43:32 -0500 Subject: [Distutils] IBM Developerworks setuptools article - example doesnt work ? Message-ID: <8C384702-3DA2-49FF-BF62-722BEDAA9CE3@zzzcomputing.com> I just read an IBM Developerworks article on setuptools today, its linked at: http://www-128.ibm.com/developerworks/linux/library/l-cppeak3.html? ca=dgr-lnxw07PythonEggWithSetuptools it refers to the ability to use pkg_resources.require() in a Python script to specify any particular version of a package that happens to exist in site-packages, overriding the currently active package...something i used to think setuptools could do, but then figured it couldnt (even though it seems like obviously useful functionality), and it seems that it still cannot. from the article's example: # install Gnosis_Utils (latest version is 1.2.1) easy_install -f http://gnosis.cx/download/Gnosis_Utils.More/ Gnosis_Utils # install version 1.2.0 of Gnosis_Utils easy_install -f http://gnosis.cx/download/Gnosis_Utils.More/ "Gnosis_Utils==1.2.0" # re-activate version 1.2.1 easy_install "Gnosis_Utils==1.2.1" # run a script that requires version 1.2.0 from pkg_resources import require require("Gnosis_Utils==1.2.0") Traceback (most recent call last): File "", line 1, in ? File "/Library/Frameworks/Python.framework/Versions/2.4/lib/ python2.4/site-packages/setuptools-0.6c3-py2.4.egg/pkg_resources.py", line 585, in require needed = self.resolve(parse_requirements(requirements)) File "/Library/Frameworks/Python.framework/Versions/2.4/lib/ python2.4/site-packages/setuptools-0.6c3-py2.4.egg/pkg_resources.py", line 487, in resolve raise VersionConflict(dist,req) # XXX put more info here pkg_resources.VersionConflict: (Gnosis-Utils 1.2.1 (/Library/ Frameworks/Python.framework/Versions/2.4/lib/python2.4/site-packages/ Gnosis_Utils-1.2.1-py2.4.egg), Requirement.parse('Gnosis-Utils==1.2.0')) bug ? or feature ? From bob at redivi.com Mon Oct 30 21:00:18 2006 From: bob at redivi.com (Bob Ippolito) Date: Mon, 30 Oct 2006 12:00:18 -0800 Subject: [Distutils] IBM Developerworks setuptools article - example doesnt work ? In-Reply-To: <8C384702-3DA2-49FF-BF62-722BEDAA9CE3@zzzcomputing.com> References: <8C384702-3DA2-49FF-BF62-722BEDAA9CE3@zzzcomputing.com> Message-ID: <6a36e7290610301200l6304d1d8qede0c1660d8ea02e@mail.gmail.com> On 10/30/06, Michael Bayer wrote: > I just read an IBM Developerworks article on setuptools today, its > linked at: > > http://www-128.ibm.com/developerworks/linux/library/l-cppeak3.html? > ca=dgr-lnxw07PythonEggWithSetuptools > > it refers to the ability to use pkg_resources.require() in a Python > script to specify any particular version of a package that happens to > exist in site-packages, overriding the currently active > package...something i used to think setuptools could do, but then > figured it couldnt (even though it seems like obviously useful > functionality), and it seems that it still cannot. from the > article's example: > > # install Gnosis_Utils (latest version is 1.2.1) > easy_install -f http://gnosis.cx/download/Gnosis_Utils.More/ > Gnosis_Utils > > # install version 1.2.0 of Gnosis_Utils > easy_install -f http://gnosis.cx/download/Gnosis_Utils.More/ > "Gnosis_Utils==1.2.0" > > # re-activate version 1.2.1 > easy_install "Gnosis_Utils==1.2.1" > > # run a script that requires version 1.2.0 > from pkg_resources import require > require("Gnosis_Utils==1.2.0") > > Traceback (most recent call last): > File "", line 1, in ? > File "/Library/Frameworks/Python.framework/Versions/2.4/lib/ > python2.4/site-packages/setuptools-0.6c3-py2.4.egg/pkg_resources.py", > line 585, in require > needed = self.resolve(parse_requirements(requirements)) > File "/Library/Frameworks/Python.framework/Versions/2.4/lib/ > python2.4/site-packages/setuptools-0.6c3-py2.4.egg/pkg_resources.py", > line 487, in resolve > raise VersionConflict(dist,req) # XXX put more info here > pkg_resources.VersionConflict: (Gnosis-Utils 1.2.1 (/Library/ > Frameworks/Python.framework/Versions/2.4/lib/python2.4/site-packages/ > Gnosis_Utils-1.2.1-py2.4.egg), Requirement.parse('Gnosis-Utils==1.2.0')) > > > bug ? or feature ? Feature. It can do multi-version installs, but only if explicitly specified. The problem is that 1.2.1 is already on sys.path, so setuptools gives up because it doesn't know if something is using Gnosis-Utils or not. If the directory *containing* the egg was on sys.path, then it would pick the exact version... but then you must *always* require it before it is imported. -bob From mike_mp at zzzcomputing.com Mon Oct 30 21:54:05 2006 From: mike_mp at zzzcomputing.com (Michael Bayer) Date: Mon, 30 Oct 2006 15:54:05 -0500 Subject: [Distutils] IBM Developerworks setuptools article - example doesnt work ? In-Reply-To: <6a36e7290610301200l6304d1d8qede0c1660d8ea02e@mail.gmail.com> References: <8C384702-3DA2-49FF-BF62-722BEDAA9CE3@zzzcomputing.com> <6a36e7290610301200l6304d1d8qede0c1660d8ea02e@mail.gmail.com> Message-ID: <10DD7A1F-3146-4FCA-940E-EA6F77BD94F1@zzzcomputing.com> On Oct 30, 2006, at 3:00 PM, Bob Ippolito wrote: > Feature. It can do multi-version installs, but only if explicitly > specified. > > The problem is that 1.2.1 is already on sys.path, so setuptools gives > up because it doesn't know if something is using Gnosis-Utils or not. > If the directory *containing* the egg was on sys.path, then it would > pick the exact version... but then you must *always* require it before > it is imported. Ah, terrific. -m allows this feature, although i wonder why -m then makes regular "import", for those scripts that just want the default version, not useable (but i have not yet searched for this answer, so feel free to ignore). Too bad a high-profile article on IBM got the details on this inaccurately. From bob at redivi.com Mon Oct 30 21:59:32 2006 From: bob at redivi.com (Bob Ippolito) Date: Mon, 30 Oct 2006 12:59:32 -0800 Subject: [Distutils] IBM Developerworks setuptools article - example doesnt work ? In-Reply-To: <10DD7A1F-3146-4FCA-940E-EA6F77BD94F1@zzzcomputing.com> References: <8C384702-3DA2-49FF-BF62-722BEDAA9CE3@zzzcomputing.com> <6a36e7290610301200l6304d1d8qede0c1660d8ea02e@mail.gmail.com> <10DD7A1F-3146-4FCA-940E-EA6F77BD94F1@zzzcomputing.com> Message-ID: <6a36e7290610301259p16d14e7btbdd93edf519ece61@mail.gmail.com> On 10/30/06, Michael Bayer wrote: > > On Oct 30, 2006, at 3:00 PM, Bob Ippolito wrote: > > Feature. It can do multi-version installs, but only if explicitly > > specified. > > > > The problem is that 1.2.1 is already on sys.path, so setuptools gives > > up because it doesn't know if something is using Gnosis-Utils or not. > > If the directory *containing* the egg was on sys.path, then it would > > pick the exact version... but then you must *always* require it before > > it is imported. > > Ah, terrific. -m allows this feature, although i wonder why -m then > makes regular "import", for those scripts that just want the default > version, not useable (but i have not yet searched for this answer, so > feel free to ignore). You're thinking about this backwards. Multi-version installs don't "make import unusable", they simply skip the step that adds the contents of the egg to sys.path. That is the only thing that differentiates a normal install and a multi-version install. The reason multi-version works at all is because it's sure that nothing in the egg is already in use by Python because it doesn't live on sys.path so none of its contents can possibly be loaded into the interpreter. Require looks for eggs and adds the best match to sys.path. -bob From pje at telecommunity.com Mon Oct 30 23:03:17 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 30 Oct 2006 17:03:17 -0500 Subject: [Distutils] IBM Developerworks setuptools article - example doesnt work ? In-Reply-To: <10DD7A1F-3146-4FCA-940E-EA6F77BD94F1@zzzcomputing.com> References: <6a36e7290610301200l6304d1d8qede0c1660d8ea02e@mail.gmail.com> <8C384702-3DA2-49FF-BF62-722BEDAA9CE3@zzzcomputing.com> <6a36e7290610301200l6304d1d8qede0c1660d8ea02e@mail.gmail.com> Message-ID: <5.1.1.6.0.20061030170035.026da830@sparrow.telecommunity.com> At 03:54 PM 10/30/2006 -0500, Michael Bayer wrote: >On Oct 30, 2006, at 3:00 PM, Bob Ippolito wrote: > > Feature. It can do multi-version installs, but only if explicitly > > specified. > > > > The problem is that 1.2.1 is already on sys.path, so setuptools gives > > up because it doesn't know if something is using Gnosis-Utils or not. > > If the directory *containing* the egg was on sys.path, then it would > > pick the exact version... but then you must *always* require it before > > it is imported. > >Ah, terrific. -m allows this feature, although i wonder why -m then >makes regular "import", for those scripts that just want the default >version, When you have multiple versions, there is no such thing as a default version. Note, however, that if you simply build your main program as a setuptools project, and define its dependencies in your setup.py, then you don't need to worry about any of this; even if there is a default version it can be overridden when the dependency is specified using setuptools. Most code shouldn't call require() at runtime; it should just declare dependencies in a setup script, making this whole question a nonissue. From mike_mp at zzzcomputing.com Mon Oct 30 23:24:35 2006 From: mike_mp at zzzcomputing.com (Michael Bayer) Date: Mon, 30 Oct 2006 17:24:35 -0500 Subject: [Distutils] IBM Developerworks setuptools article - example doesnt work ? In-Reply-To: <5.1.1.6.0.20061030170035.026da830@sparrow.telecommunity.com> References: <6a36e7290610301200l6304d1d8qede0c1660d8ea02e@mail.gmail.com> <8C384702-3DA2-49FF-BF62-722BEDAA9CE3@zzzcomputing.com> <6a36e7290610301200l6304d1d8qede0c1660d8ea02e@mail.gmail.com> <5.1.1.6.0.20061030170035.026da830@sparrow.telecommunity.com> Message-ID: <61FB2A3D-3567-49D7-AB31-628E354DA5A3@zzzcomputing.com> On Oct 30, 2006, at 5:03 PM, Phillip J. Eby wrote: > Note, however, that if you simply build your main program as a > setuptools project, and define its dependencies in your setup.py, > then you don't need to worry about any of this; even if there is a > default version it can be overridden when the dependency is > specified using setuptools. Most code shouldn't call require() at > runtime; it should just declare dependencies in a setup script, > making this whole question a nonissue. that is definitely interesting. this works independently of whether - m was ever used, correct ? From pje at telecommunity.com Mon Oct 30 23:32:51 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 30 Oct 2006 17:32:51 -0500 Subject: [Distutils] IBM Developerworks setuptools article - example doesnt work ? In-Reply-To: <61FB2A3D-3567-49D7-AB31-628E354DA5A3@zzzcomputing.com> References: <5.1.1.6.0.20061030170035.026da830@sparrow.telecommunity.com> <6a36e7290610301200l6304d1d8qede0c1660d8ea02e@mail.gmail.com> <8C384702-3DA2-49FF-BF62-722BEDAA9CE3@zzzcomputing.com> <6a36e7290610301200l6304d1d8qede0c1660d8ea02e@mail.gmail.com> <5.1.1.6.0.20061030170035.026da830@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20061030173023.03ec6dc8@sparrow.telecommunity.com> At 05:24 PM 10/30/2006 -0500, Michael Bayer wrote: >On Oct 30, 2006, at 5:03 PM, Phillip J. Eby wrote: > >>Note, however, that if you simply build your main program as a >>setuptools project, and define its dependencies in your setup.py, >>then you don't need to worry about any of this; even if there is a >>default version it can be overridden when the dependency is >>specified using setuptools. Most code shouldn't call require() at >>runtime; it should just declare dependencies in a setup script, >>making this whole question a nonissue. > >that is definitely interesting. this works independently of whether - m >was ever used, correct ? Yes. When setuptools builds a script wrapper for a program, it can adjust sys.path while Python is starting up, to remove the default versions and replace them with other versions of required libraries if necessary (as long as the required versions are installed, that is). Basically, when the pkg_resources module is first imported, it tries to satisfy all requirements using the default versions, and if there are any conflicts, it throws out all the defaults and builds up a new working set of active eggs from scratch, using available eggs from any directories on sys.path. From exarkun at divmod.com Tue Oct 31 04:18:01 2006 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Mon, 30 Oct 2006 22:18:01 -0500 Subject: [Distutils] Setuptools with custom path configuration In-Reply-To: <20061017154307.26151.509673066.divmod.quotient.4957@ohm> Message-ID: <20061031031801.20948.413486434.divmod.quotient.6730@ohm> On Tue, 17 Oct 2006 11:43:07 -0400, Jean-Paul Calderone wrote: >On Tue, 17 Oct 2006 11:32:13 -0400, "Phillip J. Eby" wrote: >> [snip] >> >>You're right, this configuration isn't really supported well. easy_install >>expects non-PYTHONPATH site dirs to be supported by Python itself, not by a >>PYTHONPATH dir causing the other one to be loaded. I'll have to see about a >>fix. >> > >Okay, thanks. Let me know if there's anything I can do to help; testing, etc. > Had a chance to do anything with this yet? >Jean-Paul From pje at telecommunity.com Tue Oct 31 07:02:35 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 31 Oct 2006 01:02:35 -0500 Subject: [Distutils] Setuptools with custom path configuration In-Reply-To: <20061031031801.20948.413486434.divmod.quotient.6730@ohm> References: <20061017154307.26151.509673066.divmod.quotient.4957@ohm> Message-ID: <5.1.1.6.0.20061031005451.029a4900@sparrow.telecommunity.com> At 10:18 PM 10/30/2006 -0500, Jean-Paul Calderone wrote: >On Tue, 17 Oct 2006 11:43:07 -0400, Jean-Paul Calderone > wrote: > >On Tue, 17 Oct 2006 11:32:13 -0400, "Phillip J. Eby" > wrote: > >> [snip] > >> > >>You're right, this configuration isn't really supported well. easy_install > >>expects non-PYTHONPATH site dirs to be supported by Python itself, not by a > >>PYTHONPATH dir causing the other one to be loaded. I'll have to see > about a > >>fix. > >> > > > >Okay, thanks. Let me know if there's anything I can do to help; > testing, etc. > > > >Had a chance to do anything with this yet? No. It's probably as simple as removing the -E from the test code, but I need to check my assumptions and make sure there isn't some other reason -E is needed. It's on my list for when I get back to work on setuptools stuff, hopefully later this week. A bunch of people at OSAF are currently blocked on other work I'm doing for them, so setuptools ATM is lower on my priority list, as is pretty much any other programming work. Of course, if somebody else wants to hazard an investigation into what, if anything, might be broken by removing the -E, ... oh crap. There's probably a heck of a lot simpler way to fix this: use the --site-dirs option to easy_install, or set it in a distutils configuration file. Duh. I should've thought of suggesting that before. Here's the bit from the docs: ``--site-dirs=DIRLIST, -S DIRLIST`` (New in 0.6a1) Specify one or more custom "site" directories (separated by commas). "Site" directories are directories where ``.pth`` files are processed, such as the main Python ``site-packages`` directory. As of 0.6a10, EasyInstall automatically detects whether a given directory processes ``.pth`` files (or can be made to do so), so you should not normally need to use this option. It is now only necessary if you want to override EasyInstall's judgment and force an installation directory to be treated as if it supported ``.pth`` files. I should probably still look into removing the -E, but this is probably a sufficient immediate workaround for your current situation. Sorry I didn't think of it before.