From joss at debian.org Mon Dec 1 01:39:51 2008 From: joss at debian.org (Josselin Mouette) Date: Mon, 01 Dec 2008 01:39:51 +0100 Subject: [Distutils] Are egg files needed when building/installing via --root? In-Reply-To: <49317E5B.4000309@sun.com> References: <492ED71F.70102@sun.com> <20081127205838.E52443A40DF@sparrow.telecommunity.com> <4930D80A.8010205@sun.com> <20081129042051.87EF43A40B2@sparrow.telecommunity.com> <670161DF-A5DA-439B-89DF-6BBEEE24076B@zooko.com> <49317E5B.4000309@sun.com> Message-ID: <1228091991.5005.67.camel@tomoyo> Le samedi 29 novembre 2008 ? 09:39 -0800, Brian Cameron a ?crit : > > I'm aware of three linux distributions that used to strip out .egg-info > > files and then changes their policy and now leave them unmolested: > > Fedora, Debian and Ubuntu. Actually we still strip out the infamous "-py2.X" suffix. (If files are different from a Python version to another, there are already better mechanisms to deal with this.) The reason we used to strip them was, IIRC, that in early versions it was not possible to ship this information as files; the package had to be shipped as an egg to work correctly. Since they can be shipped in .egg-info directories, this is not really a big issue to ship them. > Interesting, what sort of problems did the distros run into if they > removed the files? That would be the sort of information that would > be useful to add to the manual, I think. When stripping .egg-info files, you run into problems with useless code that checks for dependencies (while the packaging system already checks for them), so you have to strip such code as well. You also run into issues with packages that use egg information to look for plugins. For packages that use the easy_install insanity, this is even worse, since they can start downloading packages by mistakenly thinking their dependencies are missing. This mechanism is also disabled now, but it?s better not risking to run into it. Cheers, -- .''`. : :' : We are debian.org. Lower your prices, surrender your code. `. `' We will add your hardware and software distinctiveness to `- our own. Resistance is futile. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Ceci est une partie de message num?riquement sign?e URL: From zooko at zooko.com Mon Dec 1 04:17:14 2008 From: zooko at zooko.com (zooko) Date: Sun, 30 Nov 2008 20:17:14 -0700 Subject: [Distutils] Are egg files needed when building/installing via --root? In-Reply-To: <1228091991.5005.67.camel@tomoyo> References: <492ED71F.70102@sun.com> <20081127205838.E52443A40DF@sparrow.telecommunity.com> <4930D80A.8010205@sun.com> <20081129042051.87EF43A40B2@sparrow.telecommunity.com> <670161DF-A5DA-439B-89DF-6BBEEE24076B@zooko.com> <49317E5B.4000309@sun.com> <1228091991.5005.67.camel@tomoyo> Message-ID: <3BF2D4E0-D600-42B0-8FFD-30B9198979F6@zooko.com> On Nov 30, 2008, at 17:39 PM, Josselin Mouette wrote: > When stripping .egg-info files, you run into problems with useless > code that checks for dependencies (while the packaging system > already checks for them), so you have to strip such code as well. Let's put it like this: as the author of an application and some libraries written in Python, I would like it if my code could run on $operating_system_distribution even before someone has gotten around to packaging it for $operating_system_distribution. Therefore, I need a way to tell whether some dependency is already installed. I could write a giant switch statement for all known package management tools: if platform.dist()[0] == 'debian': os.system("dpkg --list...") elif platform.dist()[0] == 'redhat': os.system("rpm -q...") elif platform.system() == 'windows': import win32api # probe registry ??? ... etc My code currently runs on nine operating systems and I hope to support a lot more before I'm through, so this switch statement would be a significant undertaking to support my users on various platforms. Fortunately, pkg_resources does this for me in a generic, cross- platform way so that my workload is substantially reduced: pkg_resources.require('my_dependency >= 1.0') Note, this didn't use to work so well on some Linux distributions back in the bad old days when they removed .egg-info files, but nowadays it works very well on all of the aforementioned nine operating system distributions that I am aware of. Now, you as a Debian developer may feel that you would have developed my application differently than me, such as perhaps step 1: make the code run on $operating_system_distribution even before it has been packaged for $operating_system_distribution, or you may not, but either way I truly appreciate that nowadays you preserve the .egg- info files so that I have the option of doing it this way. Regards, Zooko From joss at debian.org Mon Dec 1 09:27:08 2008 From: joss at debian.org (Josselin Mouette) Date: Mon, 01 Dec 2008 09:27:08 +0100 Subject: [Distutils] Are egg files needed when building/installing via --root? In-Reply-To: <3BF2D4E0-D600-42B0-8FFD-30B9198979F6@zooko.com> References: <492ED71F.70102@sun.com> <20081127205838.E52443A40DF@sparrow.telecommunity.com> <4930D80A.8010205@sun.com> <20081129042051.87EF43A40B2@sparrow.telecommunity.com> <670161DF-A5DA-439B-89DF-6BBEEE24076B@zooko.com> <49317E5B.4000309@sun.com> <1228091991.5005.67.camel@tomoyo> <3BF2D4E0-D600-42B0-8FFD-30B9198979F6@zooko.com> Message-ID: <1228120028.5038.7.camel@tomoyo> Le dimanche 30 novembre 2008 ? 20:17 -0700, zooko a ?crit : > Let's put it like this: as the author of an application and some > libraries written in Python, I would like it if my code could run on > $operating_system_distribution even before someone has gotten around > to packaging it for $operating_system_distribution. This is a completely orthogonal issue. > Therefore, I > need a way to tell whether some dependency is already installed. I > could write a giant switch statement for all known package management > tools: > > if platform.dist()[0] == 'debian': > os.system("dpkg --list...") > elif platform.dist()[0] == 'redhat': > os.system("rpm -q...") > elif platform.system() == 'windows': > import win32api > # probe registry ??? > ... etc And that would be very wrong. Tools like dpkg and rpm are not meant to be invoked by applications at all. > Fortunately, pkg_resources does this for me in a generic, cross- > platform way so that my workload is substantially reduced: > > pkg_resources.require('my_dependency >= 1.0') This is, TTBOMK, the only framework to do it at runtime. It is much better to do such things at installation time, since it does not slow down the application when you have many of them to check, and it allows to make packages without wondering whether the application will still behave correctly at runtime. There are existing tools like pkg-config, which are not restricted to Python, and that already do things correctly. Cheers, -- .''`. : :' : We are debian.org. Lower your prices, surrender your code. `. `' We will add your hardware and software distinctiveness to `- our own. Resistance is futile. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Ceci est une partie de message num?riquement sign?e URL: From david at ar.media.kyoto-u.ac.jp Mon Dec 1 09:25:41 2008 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Mon, 01 Dec 2008 17:25:41 +0900 Subject: [Distutils] Are egg files needed when building/installing via --root? In-Reply-To: <1228120028.5038.7.camel@tomoyo> References: <492ED71F.70102@sun.com> <20081127205838.E52443A40DF@sparrow.telecommunity.com> <4930D80A.8010205@sun.com> <20081129042051.87EF43A40B2@sparrow.telecommunity.com> <670161DF-A5DA-439B-89DF-6BBEEE24076B@zooko.com> <49317E5B.4000309@sun.com> <1228091991.5005.67.camel@tomoyo> <3BF2D4E0-D600-42B0-8FFD-30B9198979F6@zooko.com> <1228120028.5038.7.camel@tomoyo> Message-ID: <49339F85.2050705@ar.media.kyoto-u.ac.jp> Josselin Mouette wrote: > There are existing tools like pkg-config, which are not restricted to > Python, and that already do things correctly. > pkg-config can't seriously be considered as a solution for python: - it is not cross platform: it does not work on windows for once (by working, I mean not only running but being usable to check for platform specific frameworks/libraries). - Assuming it did work, there would be the problem that when you install python, you don't necessarily have pkg-config. That being said, I agree that something like pkg-config, adapted to python needs, is much better than the current python scheme. cheers, David From joss at debian.org Mon Dec 1 09:58:48 2008 From: joss at debian.org (Josselin Mouette) Date: Mon, 01 Dec 2008 09:58:48 +0100 Subject: [Distutils] Are egg files needed when building/installing via --root? In-Reply-To: <49339F85.2050705@ar.media.kyoto-u.ac.jp> References: <492ED71F.70102@sun.com> <20081127205838.E52443A40DF@sparrow.telecommunity.com> <4930D80A.8010205@sun.com> <20081129042051.87EF43A40B2@sparrow.telecommunity.com> <670161DF-A5DA-439B-89DF-6BBEEE24076B@zooko.com> <49317E5B.4000309@sun.com> <1228091991.5005.67.camel@tomoyo> <3BF2D4E0-D600-42B0-8FFD-30B9198979F6@zooko.com> <1228120028.5038.7.camel@tomoyo> <49339F85.2050705@ar.media.kyoto-u.ac.jp> Message-ID: <1228121928.5038.22.camel@tomoyo> Le lundi 01 d?cembre 2008 ? 17:25 +0900, David Cournapeau a ?crit : > pkg-config can't seriously be considered as a solution for python: > - it is not cross platform: it does not work on windows for once > (by working, I mean not only running but being usable to check for > platform specific frameworks/libraries). Checking platform-specific stuff is not in the scope of pkg-config. There are other tools to do that, the most widespread being autoconf, and they should all be able to work with pkg-config. > - Assuming it did work, there would be the problem that when you > install python, you don't necessarily have pkg-config. Yes, and just like for many other things, the lack of dependencies is a serious problem on Windows. Yet it is sad to see other OSes having to adapt to tools that only exist because of this OS. > That being said, I agree that something like pkg-config, adapted to > python needs, is much better than the current python scheme. If something is missing in pkg-config (which is not trivial, since it is already very extensible), I?m pretty sure we could get the upstream developers to integrate the necessary changes. Cheers, -- .''`. : :' : We are debian.org. Lower your prices, surrender your code. `. `' We will add your hardware and software distinctiveness to `- our own. Resistance is futile. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Ceci est une partie de message num?riquement sign?e URL: From mmanns at gmx.net Mon Dec 1 10:24:00 2008 From: mmanns at gmx.net (Martin Manns) Date: Mon, 1 Dec 2008 09:24:00 +0000 (UTC) Subject: [Distutils] =?utf-8?q?Help_with_easy=5Finstall?= Message-ID: Phillip J. Eby telecommunity.com> wrote: > From reading your setup script, I don't understand why your package > needs a .pth file of its own, let alone why it's writing it directly > in the setup script. I would expect it to work correctly if you > simply delete that portion of the setup script. In fact, it works > for me that way. Without the .pth file, the libraries in the folder pyspread are not found. If I leave it out, the program fails to start (after installation finished successfully). > > One other thing you need to change if you want easy_install to handle > > dependencies: > > > > requires=['numpy (>=1.1)', 'wx (>=2.7)'], > Either change this line (or add another) saying: > > install_requires=["numpy>=1.1", "wx>=2.7"], This does not work at all for me. I do have all dependencies met (I am installing everything else via apt) but easy_install does not recognize it and throws a DistributionNotFound: wx>=2.7 *after* installation. >Also note that this line is superfluous (neither distutils nor >setuptools need it): > >> package_dir={'pyspread': 'pyspread'}, I have removed that sucessfully. > And of course, delete the entire block below; as far as I can tell, > it's not needed for distutils or setuptools. > > >import distutils.sysconfig > >try: > > pthfile = open(get_python_lib() + "/pyspread.pth", 'w') > > pthfile.write("pyspread") > > pthfile.close() > >except IOError: > > print 'Creation of ' + distutils.sysconfig.get_python_lib() + ' > > pyspread.pth > >failed.' And how do I get the pyspread.pth file created? Best Regards Martin From david at ar.media.kyoto-u.ac.jp Mon Dec 1 10:16:23 2008 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Mon, 01 Dec 2008 18:16:23 +0900 Subject: [Distutils] Are egg files needed when building/installing via --root? In-Reply-To: <1228121928.5038.22.camel@tomoyo> References: <492ED71F.70102@sun.com> <20081127205838.E52443A40DF@sparrow.telecommunity.com> <4930D80A.8010205@sun.com> <20081129042051.87EF43A40B2@sparrow.telecommunity.com> <670161DF-A5DA-439B-89DF-6BBEEE24076B@zooko.com> <49317E5B.4000309@sun.com> <1228091991.5005.67.camel@tomoyo> <3BF2D4E0-D600-42B0-8FFD-30B9198979F6@zooko.com> <1228120028.5038.7.camel@tomoyo> <49339F85.2050705@ar.media.kyoto-u.ac.jp> <1228121928.5038.22.camel@tomoyo> Message-ID: <4933AB67.3060004@ar.media.kyoto-u.ac.jp> Josselin Mouette wrote: > > Checking platform-specific stuff is not in the scope of pkg-config. > No, but you need to "natively" refer to libraries. For example, say I build a library with visual studio, can pkg-config retreive the necessary information for build/link/version ? It can't (at least it couldn't last time I checked, which was like a year ago). > > If something is missing in pkg-config (which is not trivial, since it is > already very extensible), I?m pretty sure we could get the upstream > developers to integrate the necessary changes. > Here are a few things: - being buildable with MS toolchain (without mingw or cygin) - being usable on windows "shell" (cmd.exe) so that MS can use the information. IOW, nothing works ATM. It is obvious that pkg-config is not cross-platform and only care about POSIX-like systems. Open source projects which want to be buildable with MS toolchains all use something else, like cmake, scons, etc... to check for dependencies. The only thing which would make sens IMO is a python-specific solution, which would be cross platform, but usable only for python packages, both from command line or python prompts. David From duna.swlp at gmail.com Mon Dec 1 17:43:53 2008 From: duna.swlp at gmail.com (dunia) Date: Mon, 01 Dec 2008 11:43:53 -0500 Subject: [Distutils] problem reinstalling easy_install Message-ID: <49341449.4030408@gmail.com> Hi I installed setuptools following the instructions at http://peak.telecommunity.com/DevCenter/EasyInstall...but then i realized that this package was in my repo (i'm using Debian sid). I tried to undo my previous installation by removing from /usr/bin and /usr/lib/python2.4/site-packages all setuptools packages/files, and installed python-setuptools from the repository without inconvenients. Now, when i try to use it, i get the following error: # easy_install -v ZopeSkel error: /tmp/easy_install-8zRT0t: Too many links Please, i'm stuck here, any hints will be much appreciated. PS: sorry for my english :( From robert.kern at gmail.com Mon Dec 1 18:27:26 2008 From: robert.kern at gmail.com (Robert Kern) Date: Mon, 01 Dec 2008 11:27:26 -0600 Subject: [Distutils] Help with easy_install In-Reply-To: References: Message-ID: Martin Manns wrote: > Phillip J. Eby telecommunity.com> wrote: >> From reading your setup script, I don't understand why your package >> needs a .pth file of its own, let alone why it's writing it directly >> in the setup script. I would expect it to work correctly if you >> simply delete that portion of the setup script. In fact, it works >> for me that way. > > Without the .pth file, the libraries in the folder pyspread are not found. > If I leave it out, the program fails to start (after installation finished > successfully). Rename pyspread.py to something else, then use package imports. E.g. from pyspread import _widgets from pyspread._choicebars import MainMenu, MainToolbar from pyspread._dialogs import MacroDialog, CsvImportDialog, DimensionsEntryDialog, \ AboutDialog from pyspread._interfaces import CsvImport, string_match from pyspread.config import DEBUG, LIBPREFIX, ICONPREFIX, icon_size -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From pje at telecommunity.com Mon Dec 1 18:47:54 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 01 Dec 2008 12:47:54 -0500 Subject: [Distutils] Help with easy_install In-Reply-To: References: Message-ID: <20081201174621.279DF3A40F7@sparrow.telecommunity.com> At 09:24 AM 12/1/2008 +0000, Martin Manns wrote: >Phillip J. Eby telecommunity.com> wrote: > > From reading your setup script, I don't understand why your package > > needs a .pth file of its own, let alone why it's writing it directly > > in the setup script. I would expect it to work correctly if you > > simply delete that portion of the setup script. In fact, it works > > for me that way. > >Without the .pth file, the libraries in the folder pyspread are not found. >If I leave it out, the program fails to start (after installation finished >successfully). You'll need to be more specific about what "fails to start" means. What specifically happens? What is the traceback? > > > One other thing you need to change if you want easy_install to handle > > > dependencies: > > > > > > requires=['numpy (>=1.1)', 'wx (>=2.7)'], > > > Either change this line (or add another) saying: > > > > install_requires=["numpy>=1.1", "wx>=2.7"], > >This does not work at all for me. I do have all dependencies met (I am >installing everything else via apt) but easy_install does not >recognize it and >throws a DistributionNotFound: wx>=2.7 *after* installation. Ah. Well, in that case, I guess you can leave wx out of it. I forgot that wx doesn't have egg support yet. > > And of course, delete the entire block below; as far as I can tell, > > it's not needed for distutils or setuptools. > > > > >import distutils.sysconfig > > >try: > > > pthfile = open(get_python_lib() + "/pyspread.pth", 'w') > > > pthfile.write("pyspread") > > > pthfile.close() > > >except IOError: > > > print 'Creation of ' + distutils.sysconfig.get_python_lib() + ' > > > pyspread.pth > > >failed.' > >And how do I get the pyspread.pth file created? You don't. If your program isn't working without it, it probably means your program isn't doing its imports correctly. Offhand, my guess is that you're using pyspread.pyspread as a main program, and also trying to import things from the pyspread package. That won't work correctly, because Python puts the directory containing the __main__ script as the first entry on sys.path -- meaning that when you try to import from the pyspread *package*, you instead wind up importing from the pyspread *script*. But let's see your traceback to find out what's really going on. From pje at telecommunity.com Mon Dec 1 18:50:42 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 01 Dec 2008 12:50:42 -0500 Subject: [Distutils] problem reinstalling easy_install In-Reply-To: <49341449.4030408@gmail.com> References: <49341449.4030408@gmail.com> Message-ID: <20081201174908.660003A40F7@sparrow.telecommunity.com> At 11:43 AM 12/1/2008 -0500, dunia wrote: >Hi > >I installed setuptools following the instructions at >http://peak.telecommunity.com/DevCenter/EasyInstall...but then i >realized that this package was in my repo (i'm using Debian sid). I >tried to undo my previous installation by removing from /usr/bin and >/usr/lib/python2.4/site-packages all setuptools packages/files, and >installed python-setuptools from the repository without >inconvenients. Now, when i try to use it, i get the following error: > ># easy_install -v ZopeSkel >error: /tmp/easy_install-8zRT0t: Too many links > >Please, i'm stuck here, any hints will be much appreciated. Is it the same error message each time (i.e. /tmp/easy_install-8zRT0t)? Or does the directory name change each time? Also, is there any text that appears *before* the "error:" line, but after you type in the command? From zooko at zooko.com Mon Dec 1 18:56:57 2008 From: zooko at zooko.com (zooko) Date: Mon, 1 Dec 2008 10:56:57 -0700 Subject: [Distutils] wx egg In-Reply-To: <20081201174621.279DF3A40F7@sparrow.telecommunity.com> References: <20081201174621.279DF3A40F7@sparrow.telecommunity.com> Message-ID: <88466E72-6290-453C-AFA9-5164BCB60A9F@zooko.com> On Dec 1, 2008, at 10:47 AM, Phillip J. Eby wrote: > Ah. Well, in that case, I guess you can leave wx out of it. I > forgot that wx doesn't have egg support yet. At least last February, the wx people were waiting for setuptools to provide a way to do post-install hooks: http://lists.wxwidgets.org/cgi-bin/ezmlm-cgi?11:mss: 73095:200802:hndcapodnmpbdhbnmgpn On the other hand the Enthought folks seem to have gotten pretty far without that, two years ago: https://svn.enthought.com/enthought/ticket/1178 Regards, Zooko From mmanns at gmx.net Mon Dec 1 21:58:11 2008 From: mmanns at gmx.net (Martin Manns) Date: Mon, 1 Dec 2008 20:58:11 +0000 (UTC) Subject: [Distutils] =?utf-8?q?Help_with_easy=5Finstall?= References: <20081201174621.279DF3A40F7@sparrow.telecommunity.com> Message-ID: Phillip J. Eby telecommunity.com> writes: > At 09:24 AM 12/1/2008 +0000, Martin Manns wrote: > >Phillip J. Eby telecommunity.com> wrote: > > > From reading your setup script, I don't understand why your package > > > needs a .pth file of its own, let alone why it's writing it directly > > > in the setup script. I would expect it to work correctly if you > > > simply delete that portion of the setup script. In fact, it works > > > for me that way. > > > >Without the .pth file, the libraries in the folder pyspread are not found. > >If I leave it out, the program fails to start (after installation finished > >successfully). > > You'll need to be more specific about what "fails to start" > means. What specifically happens? What is the traceback? OK I removed all files manually including the egg, removed the wx requirement from setup.py and installed again: $ cat setup.py #!/usr/bin/env python from distutils.core import setup from distutils.sysconfig import get_python_lib setup(name='pyspread', version='0.0.10b', description='A spreadsheet that accepts a pure python expression in each cell.', license='GPL v3 :: GNU General Public License', classifiers=[ 'Development Status :: 3 - Alpha', 'Intended Audience :: End Users/Desktop', ], author='Martin Manns', author_email='mmanns at gmx.net', url='http://sourceforge.net/projects/pyspread/', install_requires=["numpy>=1.1"], requires=['numpy (>=1.1)'], scripts=['pyspread/pyspread.py'], packages=['pyspread'], package_data={'pyspread': ['icons/*.png', 'icons/actions/*.png', \ 'examples/*', \ 'doc/manual.html', 'README', 'COPYING']}, ) $ su # easy_install-2.5 pyspread-0.0.10b.zip Processing pyspread-0.0.10b.zip Running pyspread-0.0.10b/setup.py -q bdist_egg --dist-dir /tmp/easy_install-bI0ZtQ/pyspread-0.0.10b/egg-dist-tmp-VNtCCf zip_safe flag not set; analyzing archive contents... Adding pyspread 0.0.10b to easy-install.pth file Installing pyspread.py script to /usr/bin Installed /usr/lib/python2.5/site-packages/pyspread-0.0.10b-py2.5.egg Processing dependencies for pyspread==0.0.10b Finished processing dependencies for pyspread==0.0.10b # exit $ pyspread.py Traceback (most recent call last): File "/usr/bin/pyspread.py", line 5, in pkg_resources.run_script('pyspread==0.0.10b', 'pyspread.py') File "/usr/lib/python2.5/site-packages/pkg_resources.py", line 448, in run_script self.require(requires)[0].run_script(script_name, ns) File "/usr/lib/python2.5/site-packages/pkg_resources.py", line 1173, in run_script exec script_code in namespace, namespace File "/usr/bin/pyspread.py", line 65, in ImportError: No module named _widgets And indeed there is no sub-folder pyspread in site-packages and therefore no module _widgets that can be found. > >And how do I get the pyspread.pth file created? > > You don't. If your program isn't working without it, it probably > means your program isn't doing its imports correctly. > > Offhand, my guess is that you're using pyspread.pyspread as a main > program, and also trying to import things from the pyspread > package. That won't work correctly, because Python puts the > directory containing the __main__ script as the first entry on > sys.path -- meaning that when you try to import from the pyspread > *package*, you instead wind up importing from the pyspread > *script*. But let's see your traceback to find out what's really going on. Hmm. Not quite, I guess. My idea of the directory structure is as follows: + Everything but the main script that is in /usr/bin resides in [...]/site-packages/pyspread + The modules are imported directly (without pyspread.[mymodule] Probably this is a bad idea. However, with the pyspread.pth file in site-packages, it works. I just made a deb that has all files in an own directory in /usr/lib/pyspread and the images etc. in /usr/share/pyspread Perhaps, this can be done with easy_install, too. Martin From pje at telecommunity.com Mon Dec 1 23:24:51 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 01 Dec 2008 17:24:51 -0500 Subject: [Distutils] Help with easy_install In-Reply-To: References: <20081201174621.279DF3A40F7@sparrow.telecommunity.com> Message-ID: <20081201222318.598B43A40F7@sparrow.telecommunity.com> At 08:58 PM 12/1/2008 +0000, Martin Manns wrote: >And indeed there is no sub-folder pyspread in site-packages and therefore no >module _widgets that can be found. That's got nothing to do with the import error. The import error is because of the sys.path issue that I described in my previous email: > > Offhand, my guess is that you're using pyspread.pyspread as a main > > program, and also trying to import things from the pyspread > > package. That won't work correctly, because Python puts the > > directory containing the __main__ script as the first entry on > > sys.path -- meaning that when you try to import from the pyspread > > *package*, you instead wind up importing from the pyspread > > *script*. ... >My idea of the directory structure is as follows: >+ Everything but the main script that is in /usr/bin >resides in [...]/site-packages/pyspread >+ The modules are imported directly (without pyspread.[mymodule] >Probably this is a bad idea. It's a *very* bad idea -- and not because of the distutils or setuptools, either. For one thing, you can end up with two copies of the same module in sys.modules under different names, i.e. both as '_foo' and 'pyspread._foo', and then you can end up with duplicate classes, making isinstance() tests fail (among other potential oddities). IOW, putting a __main__ script inside a package is bad juju where Python is concerned. Also naming that __main__ script the *same name as the package* then borders on insanity. ;-) That's why Robert Kern was saying you need to rename pyspread.py to something else, and then use package imports only. That will keep you from running afoul of the duplication problem. >However, with the pyspread.pth file in >site-packages, it works. Yeah, that's just hiding the problem, it's not really fixing it. Also, it's in very bad taste for a setup.py script to unconditionally write files; as it stands, somebody running "setup.py --help" will get a .pth file installed... and NOT to their configured --install-lib, either! (i.e., it may write to somewhere *other* than where they told the distutils to install the package). >I just made a deb that has all files in an own directory in /usr/lib/pyspread >and the images etc. in /usr/share/pyspread >Perhaps, this can be done with easy_install, too. Ah, here's where your problem is. Instead of using get_python_lib() to find the default package directory, i.e.: LIBPREFIX = distutils.sysconfig.get_python_lib() + '/pyspread/' ICONPREFIX = distutils.sysconfig.get_python_lib() + '/pyspread/' Do this instead: import os.path LIBPREFIX = ICONPREFIX = os.path.dirname(__file__) This will work with both distutils and setuptools, including cases where somebody installs your program someplace other than the system-wide site-packages directory. (E.g. using "setup.py install --install-lib=~/pylib") Also, by using __file__, setuptools will be able to notice that you're accessing your own package contents directly, and will install your package unzipped. In short, all of these changes will only improve your results and packaging, whether your users are using distutils or setuptools to install it. From cgalvan at mail.utexas.edu Tue Dec 2 03:08:34 2008 From: cgalvan at mail.utexas.edu (Chris Galvan) Date: Mon, 01 Dec 2008 20:08:34 -0600 Subject: [Distutils] wx egg In-Reply-To: <88466E72-6290-453C-AFA9-5164BCB60A9F@zooko.com> References: <20081201174621.279DF3A40F7@sparrow.telecommunity.com> <88466E72-6290-453C-AFA9-5164BCB60A9F@zooko.com> Message-ID: <493498A2.1060405@mail.utexas.edu> zooko wrote: > On Dec 1, 2008, at 10:47 AM, Phillip J. Eby wrote: > >> Ah. Well, in that case, I guess you can leave wx out of it. I >> forgot that wx doesn't have egg support yet. > > At least last February, the wx people were waiting for setuptools to > provide a way to do post-install hooks: > > http://lists.wxwidgets.org/cgi-bin/ezmlm-cgi?11:mss:73095:200802:hndcapodnmpbdhbnmgpn > > > On the other hand the Enthought folks seem to have gotten pretty far > without that, two years ago: > > https://svn.enthought.com/enthought/ticket/1178 Actually, at Enthought we have eggs on several different platforms, but they do require post-install scripts. -- Chris Galvan > > Regards, > > Zooko > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > From pje at telecommunity.com Tue Dec 2 20:36:38 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 02 Dec 2008 14:36:38 -0500 Subject: [Distutils] problem reinstalling easy_install Message-ID: <20081202193504.5B4623A40F7@sparrow.telecommunity.com> Maybe somebody else on the distutils-sig has some idea, then. I'm lost. If I had to guess though, I'd check and see if maybe your /tmp directory has too many files or directories in it -- i.e., it seems like perhaps easy_install is trying to create a temporary directory, and failing before it can do anything else. (Creating a temporary directory would be one of the very first things it does.) At 01:59 PM 12/2/2008 -0500, dunia wrote: >Phillip J. Eby wrote: >>At 11:43 AM 12/1/2008 -0500, dunia wrote: >>>Hi >>> >>>I installed setuptools following the instructions at >>>http://peak.telecommunity.com/DevCenter/EasyInstall...but then i >>>realized that this package was in my repo (i'm using Debian sid). >>>I tried to undo my previous installation by removing from /usr/bin >>>and /usr/lib/python2.4/site-packages all setuptools >>>packages/files, and installed python-setuptools from the >>>repository without inconvenients. Now, when i try to use it, i get >>>the following error: >>> >>># easy_install -v ZopeSkel >>>error: /tmp/easy_install-8zRT0t: Too many links >>> >>>Please, i'm stuck here, any hints will be much appreciated. >> >>Is it the same error message each time (i.e. >>/tmp/easy_install-8zRT0t)? Or does the directory name change each time? >The error is always ?error: /tmp/easy_install-: Too many links? >where is the part that changes. >>Also, is there any text that appears *before* the "error:" line, >>but after you type in the command? >The only text i get is the error i mentioned, there isn't any text >before or after. > >Thanks... From pje at telecommunity.com Tue Dec 2 20:36:38 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 02 Dec 2008 14:36:38 -0500 Subject: [Distutils] problem reinstalling easy_install Message-ID: <20081202193515.53BFF3A40F7@sparrow.telecommunity.com> Maybe somebody else on the distutils-sig has some idea, then. I'm lost. If I had to guess though, I'd check and see if maybe your /tmp directory has too many files or directories in it -- i.e., it seems like perhaps easy_install is trying to create a temporary directory, and failing before it can do anything else. (Creating a temporary directory would be one of the very first things it does.) At 01:59 PM 12/2/2008 -0500, dunia wrote: >Phillip J. Eby wrote: >>At 11:43 AM 12/1/2008 -0500, dunia wrote: >>>Hi >>> >>>I installed setuptools following the instructions at >>>http://peak.telecommunity.com/DevCenter/EasyInstall...but then i >>>realized that this package was in my repo (i'm using Debian sid). >>>I tried to undo my previous installation by removing from /usr/bin >>>and /usr/lib/python2.4/site-packages all setuptools >>>packages/files, and installed python-setuptools from the >>>repository without inconvenients. Now, when i try to use it, i get >>>the following error: >>> >>># easy_install -v ZopeSkel >>>error: /tmp/easy_install-8zRT0t: Too many links >>> >>>Please, i'm stuck here, any hints will be much appreciated. >> >>Is it the same error message each time (i.e. >>/tmp/easy_install-8zRT0t)? Or does the directory name change each time? >The error is always ?error: /tmp/easy_install-: Too many links? >where is the part that changes. >>Also, is there any text that appears *before* the "error:" line, >>but after you type in the command? >The only text i get is the error i mentioned, there isn't any text >before or after. > >Thanks... From Brian.Cameron at Sun.COM Tue Dec 2 20:37:01 2008 From: Brian.Cameron at Sun.COM (Brian Cameron) Date: Tue, 02 Dec 2008 13:37:01 -0600 Subject: [Distutils] Are egg files needed when building/installing via --root? In-Reply-To: <20081129225005.E218A3A40B2@sparrow.telecommunity.com> References: <492ED71F.70102@sun.com> <20081127205838.E52443A40DF@sparrow.telecommunity.com> <4930D80A.8010205@sun.com> <20081129042051.87EF43A40B2@sparrow.telecommunity.com> <670161DF-A5DA-439B-89DF-6BBEEE24076B@zooko.com> <49317E5B.4000309@sun.com> <20081129225005.E218A3A40B2@sparrow.telecommunity.com> Message-ID: <49358E5D.3060405@sun.com> Now that we have clarified that installing the egg files are useful and a good practice, I have some additional questions. Currently, on Solaris, we build most Python modules without setuptools being present on our build machine, then we install setuptools, and lastly a handful of modules which require setuptools to build (such as CSSutils, Python-Twisted, and elisa). Is this correct, or should we be making sure setuptools is installed before installing any Python modules? I wonder if some modules optionally might use setuptools if it is present and build metadata for themselves. If so, then perhaps we should be installing setuptools before installing other Python modules in general? Or is it correct just installing setuptools before installing modules which require it? Phillip: >> Interesting, what sort of problems did the distros run into if they >> removed the files? That would be the sort of information that would >> be useful to add to the manual, I think. > > Actually, I would tend to assume that randomly removing files from a > piece of software distribution would be pretty likely to produce > problems, so I don't get why people try to delete them in the first > place... or even why they'd *want* to. Here at Sun we weren't sure if the files needed to be installed when you build with "setup.py install --root". That's why I brought up the original question. It sounds like other distros have also been a bit confused in this area, so some additional documentation might help avoid similar problems going forward. At least it would provide a more official pointer when people ask questions. > Again, if you'd like to submit a documentation patch at a spot you think > would be suitable, I'll certainly look it over for inclusion. It would probably be good to have a section in the documentation which is more targeting the distros, so the interfaces (e.g. this egg-info metadata) which are installed when you build modules with setuptools are described and why these files are useful. I think the information would probably be more accurate if someone who works more closely with setuptools were to write such documentation if there is general agreement that it would be useful. I think much of the information that should go in this documentation has already been brought to light in this discussion thread, though. > Personally, I have no idea what to say or where to say it, as it sounds > too much to me like saying, "we suggest you not remove your internal > organs, as doing so may cause undesirable results." ;-) If the usefulness is that it helps with dependency checking, this would be sufficient, I'd think. Just to clarify what the best practices are when installing egg metadata for various modules would be a good addition to the documentation, I think. Brian From david at ar.media.kyoto-u.ac.jp Wed Dec 3 04:01:46 2008 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Wed, 03 Dec 2008 12:01:46 +0900 Subject: [Distutils] Are egg files needed when building/installing via --root? In-Reply-To: <49358E5D.3060405@sun.com> References: <492ED71F.70102@sun.com> <20081127205838.E52443A40DF@sparrow.telecommunity.com> <4930D80A.8010205@sun.com> <20081129042051.87EF43A40B2@sparrow.telecommunity.com> <670161DF-A5DA-439B-89DF-6BBEEE24076B@zooko.com> <49317E5B.4000309@sun.com> <20081129225005.E218A3A40B2@sparrow.telecommunity.com> <49358E5D.3060405@sun.com> Message-ID: <4935F69A.90207@ar.media.kyoto-u.ac.jp> Brian Cameron wrote: > > Now that we have clarified that installing the egg files are useful > and a good practice, I have some additional questions. > > Currently, on Solaris, we build most Python modules without setuptools > being present on our build machine, then we install setuptools, and > lastly a handful of modules which require setuptools to build (such as > CSSutils, Python-Twisted, and elisa). Is this correct, or should we > be making sure setuptools is installed before installing any Python > modules? > > I wonder if some modules optionally might use setuptools if it is present > and build metadata for themselves. Yes, some setup.py files look like this: try: from setuptools import .... except ImportError: from distutils import ... Most linux distributions do not use setuptools for the packages (if a package requires setuptools, it is often built with --single-version-externally-managed --record=somefile, meaning standard distutils behavior - note that in both cases, the egg.info is produced, and somewhat independent of setuptools). Setuptools brings nothing for systems with a dependency system. Unless there is a strong reason to do otherwise, I would say it is better to follow linux distributions practice. I can't find it ATM, but they are somewhat documented somewhere on the ML, by some Fedora and Debian packagers. David From pje at telecommunity.com Wed Dec 3 05:52:02 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 02 Dec 2008 23:52:02 -0500 Subject: [Distutils] Are egg files needed when building/installing via --root? In-Reply-To: <4935F69A.90207@ar.media.kyoto-u.ac.jp> References: <492ED71F.70102@sun.com> <20081127205838.E52443A40DF@sparrow.telecommunity.com> <4930D80A.8010205@sun.com> <20081129042051.87EF43A40B2@sparrow.telecommunity.com> <670161DF-A5DA-439B-89DF-6BBEEE24076B@zooko.com> <49317E5B.4000309@sun.com> <20081129225005.E218A3A40B2@sparrow.telecommunity.com> <49358E5D.3060405@sun.com> <4935F69A.90207@ar.media.kyoto-u.ac.jp> Message-ID: <20081203045029.2598B3A40DF@sparrow.telecommunity.com> At 12:01 PM 12/3/2008 +0900, David Cournapeau wrote: >Setuptools brings nothing for systems with a dependency system. To the system, no; to the package, yes. Please remember that .egg-info isn't just for dependencies, it's also for entry points (an inter-package communication mechanism, so that one package can provide plugins for another). From zooko at zooko.com Wed Dec 3 13:31:44 2008 From: zooko at zooko.com (zooko) Date: Wed, 3 Dec 2008 05:31:44 -0700 Subject: [Distutils] Are egg files needed when building/installing via --root? In-Reply-To: <4935F69A.90207@ar.media.kyoto-u.ac.jp> References: <492ED71F.70102@sun.com> <20081127205838.E52443A40DF@sparrow.telecommunity.com> <4930D80A.8010205@sun.com> <20081129042051.87EF43A40B2@sparrow.telecommunity.com> <670161DF-A5DA-439B-89DF-6BBEEE24076B@zooko.com> <49317E5B.4000309@sun.com> <20081129225005.E218A3A40B2@sparrow.telecommunity.com> <49358E5D.3060405@sun.com> <4935F69A.90207@ar.media.kyoto-u.ac.jp> Message-ID: <2E0A1308-1D14-4738-85E9-02E476F59D5B@zooko.com> On Dec 2, 2008, at 20:01 PM, David Cournapeau wrote: > Unless there is a strong reason to do otherwise, I would say it is > better to follow linux distributions practice. I can't find it ATM, > but they are somewhat documented somewhere on the ML, by some > Fedora and Debian packagers. The Fedora policy is a good one: http://fedoraproject.org/wiki/Packaging/Python/Eggs Regards, Zooko --- http://allmydata.org -- Tahoe, the Least-Authority Filesystem http://allmydata.com -- back up all your files for $10/month From fadhley.salim at uk.calyon.com Thu Dec 4 11:17:30 2008 From: fadhley.salim at uk.calyon.com (Fadhley Salim) Date: Thu, 4 Dec 2008 10:17:30 -0000 Subject: [Distutils] Binary only egg References: <492ED71F.70102@sun.com> <20081127205838.E52443A40DF@sparrow.telecommunity.com> <4930D80A.8010205@sun.com> <20081129042051.87EF43A40B2@sparrow.telecommunity.com> <670161DF-A5DA-439B-89DF-6BBEEE24076B@zooko.com> <49317E5B.4000309@sun.com> <20081129225005.E218A3A40B2@sparrow.telecommunity.com><49358E5D.3060405@sun.com> <4935F69A.90207@ar.media.kyoto-u.ac.jp> <2E0A1308-1D14-4738-85E9-02E476F59D5B@zooko.com> Message-ID: <7F347D91614EBC48AA7540A2A76D3BB204483E95@MXCU10MX1.MSX.CIB> Could somebody point me to the distutils option which prevents built eggs from containing the source-code: The egg should only contain the compiled byte-code. I seem to remember that there's a simple keyword argument to the setuptools.setup command which will do this. Thanks! -------------- next part -------------- This email does not create a legal relationship between any member of the Cr?dit Agricole group and the recipient or constitute investment advice. The content of this email should not be copied or disclosed (in whole or part) to any other person. It may contain information which is confidential, privileged or otherwise protected from disclosure. If you are not the intended recipient, you should notify us and delete it from your system. Emails may be monitored, are not secure and may be amended, destroyed or contain viruses and in communicating with us such conditions are accepted. Any content which does not relate to business matters is not endorsed by us. Calyon is authorised by the Comit? des Etablissements de Cr?dit et des Entreprises d'Investissement (CECEI) and supervised by the Commission Bancaire in France and subject to limited regulation by the Financial Services Authority. Details about the extent of our regulation by the Financial Services Authority are available from us on request. Calyon is incorporated in France with limited liability and registered in England & Wales. Registration number: FC008194. Registered office: Broadwalk House, 5 Appold Street, London, EC2A 2DA. From pje at telecommunity.com Fri Dec 5 00:44:04 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 04 Dec 2008 18:44:04 -0500 Subject: [Distutils] Binary only egg In-Reply-To: <7F347D91614EBC48AA7540A2A76D3BB204483E95@MXCU10MX1.MSX.CIB > References: <492ED71F.70102@sun.com> <20081127205838.E52443A40DF@sparrow.telecommunity.com> <4930D80A.8010205@sun.com> <20081129042051.87EF43A40B2@sparrow.telecommunity.com> <670161DF-A5DA-439B-89DF-6BBEEE24076B@zooko.com> <49317E5B.4000309@sun.com> <20081129225005.E218A3A40B2@sparrow.telecommunity.com> <49358E5D.3060405@sun.com> <4935F69A.90207@ar.media.kyoto-u.ac.jp> <2E0A1308-1D14-4738-85E9-02E476F59D5B@zooko.com> <7F347D91614EBC48AA7540A2A76D3BB204483E95@MXCU10MX1.MSX.CIB> Message-ID: <20081204234230.06E4E3A40DF@sparrow.telecommunity.com> At 10:17 AM 12/4/2008 +0000, Fadhley Salim wrote: >Could somebody point me to the distutils option which prevents built >eggs from containing the source-code: The egg should only contain the >compiled byte-code. I seem to remember that there's a simple keyword >argument to the setuptools.setup command which will do this. $ python setup.py bdist_egg --help Common commands: (see '--help-commands' for more) setup.py build will build the package underneath 'build/' setup.py install will install the package Global options: --verbose (-v) run verbosely (default) --quiet (-q) run quietly (turns verbosity off) --dry-run (-n) don't actually do anything --help (-h) show detailed help message Options for 'bdist_egg' command: --bdist-dir (-b) temporary directory for creating the distribution --plat-name (-p) platform name to embed in generated filenames (default: win32) --exclude-source-files remove all .py files from the generated egg --keep-temp (-k) keep the pseudo-installation tree around after creating the distribution archive --dist-dir (-d) directory to put final built distributions in --skip-build skip rebuilding everything (for testing/debugging) From zooko at zooko.com Fri Dec 5 04:43:39 2008 From: zooko at zooko.com (zooko) Date: Thu, 4 Dec 2008 20:43:39 -0700 Subject: [Distutils] ?t=info Message-ID: <774F89D3-A54A-4EE3-88EF-6E8B7CF8AFA7@zooko.com> Folks: I pointed easy_install at this server: http://allmydata.org/trac/tahoe/wiki/PackageMirror This is very cool because the files are stored on a secure, decentralized storage grid. (Details on http://allmydata.org , especially http://allmydata.org/~zooko/lafs.pdf .) However, sometimes easy_install issues an HTTP GET request for a file with "?t=info" tacked onto the end. http://pypi.python.org and apache ignore that query arg and just serve up the file, but the tahoe web server just so happens to use ?t=info for something else (the "More Info" links on the directory listing there), so this request fails to serve up the file. My question is: why does easy_install append "?t=info" to the end of those requests? Regards, Zooko --- http://allmydata.org -- Tahoe, the Least-Authority Filesystem http://allmydata.com -- back up all your files for $10/month From zooko at zooko.com Fri Dec 5 04:57:58 2008 From: zooko at zooko.com (zooko) Date: Thu, 4 Dec 2008 20:57:58 -0700 Subject: [Distutils] ?t=info In-Reply-To: <774F89D3-A54A-4EE3-88EF-6E8B7CF8AFA7@zooko.com> References: <774F89D3-A54A-4EE3-88EF-6E8B7CF8AFA7@zooko.com> Message-ID: <9ABDC3AB-ACB0-4236-B9DE-632394F5C94C@zooko.com> [following-up to my own post] Well, I feel pretty silly. The answer to my question of "Why does easy_install append ?t=info to these file URLs?" is "Because it is following the hyperlinks on your page!". I am too tired tonight to figure out how to hack tahoe and/or easy_install so that it finds the eggs on that page and doesn't stop with "Unexpected HTML page found" after following the other links. Maybe tomorrow! Regards, Zooko From mmanns at gmx.net Fri Dec 5 20:16:19 2008 From: mmanns at gmx.net (Martin Manns) Date: Fri, 5 Dec 2008 19:16:19 +0000 (UTC) Subject: [Distutils] =?utf-8?q?Help_with_easy=5Finstall?= References: <20081201174621.279DF3A40F7@sparrow.telecommunity.com> Message-ID: Phillip J. Eby telecommunity.com> writes: > Ah, here's where your problem is. Instead of using get_python_lib() > to find the default package directory, i.e.: > > LIBPREFIX = distutils.sysconfig.get_python_lib() + '/pyspread/' > ICONPREFIX = distutils.sysconfig.get_python_lib() + '/pyspread/' > > Do this instead: > > import os.path > LIBPREFIX = ICONPREFIX = os.path.dirname(__file__) This works great. Thank you for all your help (also to Robert Kern). Best Regards Martin From regebro at gmail.com Sat Dec 6 17:35:59 2008 From: regebro at gmail.com (Lennart Regebro) Date: Sat, 6 Dec 2008 17:35:59 +0100 Subject: [Distutils] Setuptools + buildout for Zope 3? Message-ID: <319e029f0812060835j218db89du7414291aeff1b522@mail.gmail.com> Hiya! What are the plans for Python 3 support for setuptools and zc.buildout? I looked through the archives quickly and couldn't see anything. If none, does anybody mind if I have a go at making 3.0 support? In fact, I already had a go this August, as a quick test, so I think I have an idea of what I'm up against. ;-) -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64 From jim at zope.com Sat Dec 6 18:48:04 2008 From: jim at zope.com (Jim Fulton) Date: Sat, 6 Dec 2008 12:48:04 -0500 Subject: [Distutils] Setuptools + buildout for Zope 3? In-Reply-To: <319e029f0812060835j218db89du7414291aeff1b522@mail.gmail.com> References: <319e029f0812060835j218db89du7414291aeff1b522@mail.gmail.com> Message-ID: On Dec 6, 2008, at 11:35 AM, Lennart Regebro wrote: > Hiya! What are the plans for Python 3 support for setuptools and > zc.buildout? I looked through the archives quickly and couldn't see > anything. > > If none, does anybody mind if I have a go at making 3.0 support? In > fact, I already had a go this August, as a quick test, so I think I > have an idea of what I'm up against. ;-) As soon as setuptools supports Python 3, then I'd want zc.buildout to. :) Jim -- Jim Fulton Zope Corporation From regebro at gmail.com Sun Dec 7 13:03:03 2008 From: regebro at gmail.com (Lennart Regebro) Date: Sun, 7 Dec 2008 13:03:03 +0100 Subject: [Distutils] Setuptools + buildout for Zope 3? In-Reply-To: References: <319e029f0812060835j218db89du7414291aeff1b522@mail.gmail.com> Message-ID: <319e029f0812070403o6992cd79k9fdd58e9ca9f8321@mail.gmail.com> On Sat, Dec 6, 2008 at 18:48, Jim Fulton wrote: > > On Dec 6, 2008, at 11:35 AM, Lennart Regebro wrote: > >> Hiya! What are the plans for Python 3 support for setuptools and >> zc.buildout? I looked through the archives quickly and couldn't see >> anything. >> >> If none, does anybody mind if I have a go at making 3.0 support? In >> fact, I already had a go this August, as a quick test, so I think I >> have an idea of what I'm up against. ;-) > > > As soon as setuptools supports Python 3, then I'd want zc.buildout to. :) Yup, I think I can get you that, but I'd need write access to the setuptools svn first. :-) -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64 From ziade.tarek at gmail.com Sun Dec 7 13:08:58 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sun, 7 Dec 2008 13:08:58 +0100 Subject: [Distutils] Setuptools + buildout for Zope 3? In-Reply-To: <319e029f0812070403o6992cd79k9fdd58e9ca9f8321@mail.gmail.com> References: <319e029f0812060835j218db89du7414291aeff1b522@mail.gmail.com> <319e029f0812070403o6992cd79k9fdd58e9ca9f8321@mail.gmail.com> Message-ID: <94bdd2610812070408t65bd7da1of3d84aa2c7593c79@mail.gmail.com> On Sun, Dec 7, 2008 at 1:03 PM, Lennart Regebro wrote: > On Sat, Dec 6, 2008 at 18:48, Jim Fulton wrote: >> >> On Dec 6, 2008, at 11:35 AM, Lennart Regebro wrote: >> >>> Hiya! What are the plans for Python 3 support for setuptools and >>> zc.buildout? I looked through the archives quickly and couldn't see >>> anything. >>> >>> If none, does anybody mind if I have a go at making 3.0 support? In >>> fact, I already had a go this August, as a quick test, so I think I >>> have an idea of what I'm up against. ;-) >> >> >> As soon as setuptools supports Python 3, then I'd want zc.buildout to. :) > > Yup, I think I can get you that, but I'd need write access to the > setuptools svn first. :-) you can post a patch here: http://bugs.python.org/setuptools/ > > -- > Lennart Regebro: Zope and Plone consulting. > http://www.colliberty.com/ > +33 661 58 14 64 > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ From setuptools at bugs.python.org Sun Dec 7 14:05:33 2008 From: setuptools at bugs.python.org (lregebro) Date: Sun, 07 Dec 2008 13:05:33 +0000 Subject: [Distutils] [issue56] [PATCH] python setup.py test in python2.6 In-Reply-To: <1228655133.9.0.923335853479.issue56@psf.upfronthosting.co.za> Message-ID: <1228655133.9.0.923335853479.issue56@psf.upfronthosting.co.za> New submission from lregebro : In Python 2.6, the default of the testRunner parameter when calling unittest.main is no longer None, but is instead TextTestRunner. Calling it with None explicitly says to use None as a test runner, so that will fail, meaning you can't run "python2.6 setup.py test", instead you get a AttributeError: 'NoneType' object has no attribute 'run' in unittest.py line 861. There are two solutions, either set the runner variable to TextTestRunner for versions > 2.5, or not pass in the parameter when it is None. I'm not sure which one is seen as neater, so I'll provide both at patches, version A and B. ---------- messages: 216 nosy: lregebro priority: bug status: unread title: [PATCH] python setup.py test in python2.6 _______________________________________________ Setuptools tracker _______________________________________________ From regebro at gmail.com Sun Dec 7 14:27:57 2008 From: regebro at gmail.com (Lennart Regebro) Date: Sun, 7 Dec 2008 14:27:57 +0100 Subject: [Distutils] Setuptools + buildout for Zope 3? In-Reply-To: <94bdd2610812070408t65bd7da1of3d84aa2c7593c79@mail.gmail.com> References: <319e029f0812060835j218db89du7414291aeff1b522@mail.gmail.com> <319e029f0812070403o6992cd79k9fdd58e9ca9f8321@mail.gmail.com> <94bdd2610812070408t65bd7da1of3d84aa2c7593c79@mail.gmail.com> Message-ID: <319e029f0812070527k3a445243y3681e9f06a0d9397@mail.gmail.com> On Sun, Dec 7, 2008 at 13:08, Tarek Ziad? wrote: > you can post a patch here: http://bugs.python.org/setuptools/ OK; so I'll do that. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64 From chris at simplistix.co.uk Fri Dec 5 17:10:21 2008 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 05 Dec 2008 16:10:21 +0000 Subject: [Distutils] setuptools/buildout: why only one index server configurable In-Reply-To: <492E941F.7010505@zopyx.com> References: <492CF953.9000705@zopyx.com> <492E937A.6090204@simplistix.co.uk> <492E941F.7010505@zopyx.com> Message-ID: <4939526D.9060108@simplistix.co.uk> Andreas Jung wrote: >> Is there some reason adding find-links with your other indexes in >> doesn't work? It works for me in the same situation... > > The index package for a complete PyPI mirror is pretty long (too long to > be loaded and to be parsed). Why do you need a complete mirror? Surely your index page only needs to list your private packages? > In addition we would have to update the > index page for our internal egg server upon every change. Well yeah, but you can just use Apache's file index or (as I do) a subversion http server for this... > right now but it sux. Using 'index' is appearently much cleaner and > resource-friendly. Really? Surely it still has to do just the same html processing, etc? Can someone please comment on this authoritatively? cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk From chris at simplistix.co.uk Fri Dec 5 17:11:40 2008 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 05 Dec 2008 16:11:40 +0000 Subject: [Distutils] Binary only egg In-Reply-To: <7F347D91614EBC48AA7540A2A76D3BB204483E95@MXCU10MX1.MSX.CIB> References: <492ED71F.70102@sun.com> <20081127205838.E52443A40DF@sparrow.telecommunity.com> <4930D80A.8010205@sun.com> <20081129042051.87EF43A40B2@sparrow.telecommunity.com> <670161DF-A5DA-439B-89DF-6BBEEE24076B@zooko.com> <49317E5B.4000309@sun.com> <20081129225005.E218A3A40B2@sparrow.telecommunity.com><49358E5D.3060405@sun.com> <4935F69A.90207@ar.media.kyoto-u.ac.jp> <2E0A1308-1D14-4738-85E9-02E476F59D5B@zooko.com> <7F347D91614EBC48AA7540A2A76D3BB204483E95@MXCU10MX1.MSX.CIB> Message-ID: <493952BC.9000301@simplistix.co.uk> Fadhley Salim wrote: > Could somebody point me to the distutils option which prevents built > eggs from containing the source-code: The egg should only contain the > compiled byte-code. I seem to remember that there's a simple keyword > argument to the setuptools.setup command which will do this. Sal, This sounds like you might be trying to prevent people from reading your source ;-) These are not the droids you're looking for... python makes this damned near impossible. Turning a .pyc back into a .py is not that hard... Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk From chris at simplistix.co.uk Fri Dec 5 17:14:34 2008 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 05 Dec 2008 16:14:34 +0000 Subject: [Distutils] bootstrap.py doesn't work offline? In-Reply-To: References: <49255B6E.2090306@simplistix.co.uk> <20081120150918.GA9399@fridge.pov.lt> Message-ID: <4939536A.8040900@simplistix.co.uk> Tarek Ziade wrote: > Since bootstrap is also used to create directories inside the buildout > or simply, to link your buildout to another > interpreter, we wanted it to work with or without an internet > connection. (because you might want to run it again when you deploy > a source buildout at your customer in an existing python environment) Indeed, any chance you could merge these changes back into "the definitive" bootstrap.py? Configurable location for the eggs would be good, would be great if it could respect [buildout] egg-directory... cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk From ziade.tarek at gmail.com Mon Dec 8 12:09:37 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Mon, 8 Dec 2008 12:09:37 +0100 Subject: [Distutils] bootstrap.py doesn't work offline? In-Reply-To: <4939536A.8040900@simplistix.co.uk> References: <49255B6E.2090306@simplistix.co.uk> <20081120150918.GA9399@fridge.pov.lt> <4939536A.8040900@simplistix.co.uk> Message-ID: <94bdd2610812080309p450108aapaa78fb33ceb102c6@mail.gmail.com> yes I could, I'll submit it to Jim I am also working on a relocatable compiled buildout right now, more info coming up this week Cheers On 12/5/08, Chris Withers wrote: > Tarek Ziade wrote: >> Since bootstrap is also used to create directories inside the buildout >> or simply, to link your buildout to another >> interpreter, we wanted it to work with or without an internet >> connection. (because you might want to run it again when you deploy >> a source buildout at your customer in an existing python environment) > > Indeed, any chance you could merge these changes back into "the > definitive" bootstrap.py? > > Configurable location for the eggs would be good, would be great if it > could respect [buildout] egg-directory... > > cheers, > > Chris > > -- > Simplistix - Content Management, Zope & Python Consulting > - http://www.simplistix.co.uk > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ From regebro at gmail.com Mon Dec 8 20:39:38 2008 From: regebro at gmail.com (Lennart Regebro) Date: Mon, 8 Dec 2008 20:39:38 +0100 Subject: [Distutils] Setuptools + buildout for Zope 3? In-Reply-To: <319e029f0812070527k3a445243y3681e9f06a0d9397@mail.gmail.com> References: <319e029f0812060835j218db89du7414291aeff1b522@mail.gmail.com> <319e029f0812070403o6992cd79k9fdd58e9ca9f8321@mail.gmail.com> <94bdd2610812070408t65bd7da1of3d84aa2c7593c79@mail.gmail.com> <319e029f0812070527k3a445243y3681e9f06a0d9397@mail.gmail.com> Message-ID: <319e029f0812081139t7b5ff952x20bf6a92cb332b4f@mail.gmail.com> Well, I tried porting disutils to Python 3, and I got all the test running, but somewhere along the path, I broke things, so now I get the error zipimport.ZipImportError: not a Zip file: 'dist/setuptools-0.7a1dev_r66608-py2.5.egg' When I try to actually *install* it. This I get on all versions of Python, so I must have broken something when I changed the code so that 2to3 would convert it happily. However, in the path of doing this, I also have found bugs in 2to3, which means that I have to massage some of the files by hand after the 2to3 conversion. So before we can really start supporting Python 3 we need to get those bugs fixed anyway. So that's the next part of this project. I have however created a diff file with changes for distutils to stop 2.6 complaining when you run "python2.6 -3 setup.py test". I'll create a bugreport with that diff soon. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64 From ziade.tarek at gmail.com Tue Dec 9 02:28:59 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 9 Dec 2008 02:28:59 +0100 Subject: [Distutils] bootstrap.py doesn't work offline? In-Reply-To: <94bdd2610812080309p450108aapaa78fb33ceb102c6@mail.gmail.com> References: <49255B6E.2090306@simplistix.co.uk> <20081120150918.GA9399@fridge.pov.lt> <4939536A.8040900@simplistix.co.uk> <94bdd2610812080309p450108aapaa78fb33ceb102c6@mail.gmail.com> Message-ID: <94bdd2610812081728j74d4d705s4bd07fb3e3ac291e@mail.gmail.com> On Mon, Dec 8, 2008 at 12:09 PM, Tarek Ziad? wrote: > yes I could, I'll submit it to Jim > > I am also working on a relocatable compiled buildout right now, > more info coming up this week Here's a first shot : http://tarekziade.wordpress.com/2008/12/09/how-to-make-binary-distribution-of-buildouts/ > > Cheers > > On 12/5/08, Chris Withers wrote: >> Tarek Ziade wrote: >>> Since bootstrap is also used to create directories inside the buildout >>> or simply, to link your buildout to another >>> interpreter, we wanted it to work with or without an internet >>> connection. (because you might want to run it again when you deploy >>> a source buildout at your customer in an existing python environment) >> >> Indeed, any chance you could merge these changes back into "the >> definitive" bootstrap.py? >> >> Configurable location for the eggs would be good, would be great if it >> could respect [buildout] egg-directory... >> >> cheers, >> >> Chris >> >> -- >> Simplistix - Content Management, Zope & Python Consulting >> - http://www.simplistix.co.uk >> >> _______________________________________________ >> Distutils-SIG maillist - Distutils-SIG at python.org >> http://mail.python.org/mailman/listinfo/distutils-sig >> > > > -- > Tarek Ziad? | Association AfPy | www.afpy.org > Blog FR | http://programmation-python.org > Blog EN | http://tarekziade.wordpress.com/ > -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ From p.f.moore at gmail.com Tue Dec 9 17:21:53 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 9 Dec 2008 16:21:53 +0000 Subject: [Distutils] Bootstrapping setuptools for Python 2.6 (Windows) Message-ID: <79990c6b0812090821w4ba6e83bhab88d2cab1c6df85@mail.gmail.com> The setuptools download page says that you should install setuptools from the provided .exe file on Windows. However, for Python 2.6, there is only a .egg file. Where should I go to get a Python 2.6 setuptools .exe installer? Thanks, Paul. From pje at telecommunity.com Tue Dec 9 18:27:01 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 09 Dec 2008 12:27:01 -0500 Subject: [Distutils] Bootstrapping setuptools for Python 2.6 (Windows) In-Reply-To: <79990c6b0812090821w4ba6e83bhab88d2cab1c6df85@mail.gmail.co m> References: <79990c6b0812090821w4ba6e83bhab88d2cab1c6df85@mail.gmail.com> Message-ID: <20081209172524.7F14B3A405E@sparrow.telecommunity.com> At 04:21 PM 12/9/2008 +0000, Paul Moore wrote: >The setuptools download page says that you should install setuptools >from the provided .exe file on Windows. However, for Python 2.6, there >is only a .egg file. Where should I go to get a Python 2.6 setuptools >.exe installer? I'd suggest downloading the source and running "python setup.py bdist_wininst" to make one. From Benedikt.Naessens at niko.be Wed Dec 10 16:59:22 2008 From: Benedikt.Naessens at niko.be (Benedikt.Naessens at niko.be) Date: Wed, 10 Dec 2008 16:59:22 +0100 Subject: [Distutils] Newbie question : Use of external packages + re-use of other code Message-ID: I want to make a Windows installer - using distutils - for the Python code I have made so far. Related to this, I have some questions about the "best way" to do this. The project is a test platform that should be used for internal use in our company. What are my problems ? * I am using Nose, PyQt etc. : how do I make sure that these packages are installed with my installer ? The main problem is here that most of the PC's in the company are behind a firewall that not allows them to download these packages from the internet (internet browsers are handled by a remote desktop) * Maybe related to the previous issue : I have written a plugin for Nose; this plugin needs to be "registered" with Nose (manually I do this with easy_install); can I do this also with distutils ? * I am also using SWIG for some code written in C++. I have already read the documentation of distutils where they advise me to build my code with distutils. Problem is that this C++ project is already rather large and that the .pyd is already generated (so I'd rather would not like to declare all my .i and .cpp files in a setup.py file). Do I have to include the generated .pyd, .lib and .py as "package data" into setup.py or are there better ways ? Maybe this sounds trivial to a lot of you, but as I am using Python for 1.5 months, I'd like to have some help from specialists ... Thanks in advance ! Kind regards, Benedikt Naessens. Disclaimer This e-mail and its attachments is intended only for the person(s) or entity to which it is addressed. If you receive this e-mail by mistake, please delete this e-mail from your system and destroy all copies of it. It may contain confidential and/or privileged information. You should not copy it or use it for any purpose nor disclose its contents to any person unless allowed by a written document between the sender and the addressee. -------------- next part -------------- An HTML attachment was scrubbed... URL: From Benedikt.Naessens at niko.be Fri Dec 12 11:13:10 2008 From: Benedikt.Naessens at niko.be (Benedikt.Naessens at niko.be) Date: Fri, 12 Dec 2008 11:13:10 +0100 Subject: [Distutils] Can Distutils handle this ? Message-ID: Ok, Maybe I will make the question shorter : is distutils the right tool for the problems described below ? Or should I rather use a tool like InnoSetup or a Windows installer (made in .NET) ? Benedikt Naessens. I want to make a Windows installer - using distutils - for the Python code I have made so far. Related to this, I have some questions about the "best way" to do this. The project is a test platform that should be used for internal use in our company. What are my problems ? * I am using Nose, PyQt etc. : how do I make sure that these packages are installed with my installer ? The main problem is here that most of the PC's in the company are behind a firewall that not allows them to download these packages from the internet (internet browsers are handled by a remote desktop) * Maybe related to the previous issue : I have written a plugin for Nose; this plugin needs to be "registered" with Nose (manually I do this with easy_install); can I do this also with distutils ? * I am also using SWIG for some code written in C++. I have already read the documentation of distutils where they advise me to build my code with distutils. Problem is that this C++ project is already rather large and that the .pyd is already generated (so I'd rather would not like to declare all my .i and .cpp files in a setup.py file). Do I have to include the generated .pyd, .lib and .py as "package data" into setup.py or are there better ways ? Maybe this sounds trivial to a lot of you, but as I am using Python for 1.5 months, I'd like to have some help from specialists ... Thanks in advance ! Benedikt Naessens. Disclaimer This e-mail and its attachments is intended only for the person(s) or entity to which it is addressed. If you receive this e-mail by mistake, please delete this e-mail from your system and destroy all copies of it. It may contain confidential and/or privileged information. You should not copy it or use it for any purpose nor disclose its contents to any person unless allowed by a written document between the sender and the addressee. -------------- next part -------------- An HTML attachment was scrubbed... URL: From david at ar.media.kyoto-u.ac.jp Fri Dec 12 11:26:04 2008 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Fri, 12 Dec 2008 19:26:04 +0900 Subject: [Distutils] Can Distutils handle this ? In-Reply-To: References: Message-ID: <49423C3C.9040708@ar.media.kyoto-u.ac.jp> Benedikt.Naessens at niko.be wrote: > > Ok, > > Maybe I will make the question shorter : is distutils the right tool > for the problems described below ? Or should I rather use a tool like > InnoSetup or a Windows installer (made in .NET) ? > The install process of distutils is extremely cumbersome, and not easy to control; if you want to install things outside your own package (libraries which are not built with distutils, etc...), you will have a hard time. I think you should use an installer, indeed, David From chris at simplistix.co.uk Fri Dec 12 13:24:31 2008 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 12 Dec 2008 12:24:31 +0000 Subject: [Distutils] problems with PyPI? Message-ID: <494257FF.7050407@simplistix.co.uk> Hi All, Is anyone else having problems with PyPI? I tried mailing the catalog-sig, but my message hasn't even shown up in the archives: http://mail.python.org/pipermail/catalog-sig/ Looks like ReST isn't rendering for package descriptions, I've recently uploaded new versions of these: http://pypi.python.org/pypi/testfixtures http://pypi.python.org/pypi/mailinglogger Also, the search and front-page-newest-packages appear to be broken... Am I doing something wrong or is PyPI hosed? cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk From stephen.pascoe at stfc.ac.uk Fri Dec 12 13:34:55 2008 From: stephen.pascoe at stfc.ac.uk (Pascoe, S (Stephen)) Date: Fri, 12 Dec 2008 12:34:55 -0000 Subject: [Distutils] [zc.buildout] distutils "scripts" keyword Message-ID: I'm coming across a lot of packages that use the distutils "scripts" keyword to identify scripts (rather than the console_scripts entry-point). Unfortunately these don't work with buildout, although they are installed with easy_install. I think this should be a solvable problem with a recipe that extends zc.recipe.egg. For instance buildbot creates the "buildbot" script. When I create a buildout with: ===================== [buildout] parts = buildbot [buildbot] recipe = zc.recipe.egg interpreter = python ===================== I can then run the script with $ bin/python -c "import pkg_resources; pkg_resources.working_set.run_script('buildbot', 'buildbot')" although I can't send it any arguments. Also the following hand-crafted script works: ================== #!/usr/bin/python import sys sys.path[0:0] = [ '/crypt/home/spascoe/expt/buildbot/eggs/buildbot-0.7.9-py2.5.egg', '/crypt/home/spascoe/expt/buildbot/eggs/Twisted-8.1.0-py2.5-linux-i686.e gg', '/crypt/home/spascoe/expt/buildbot/eggs/zope.interface-3.5.0-py2.5-linux -i686.egg', '/usr/lib/python2.5/site-packages/setuptools-0.6c9-py2.5.egg', ] import pkg_resources pkg_resources.working_set.run_script('buildbot', 'buildbot') ================== Can anyone think of a catch that would make this not work in general? Cheers, Stephen. --- Stephen Pascoe +44 (0)1235 445980 British Atmospheric Data Centre Rutherford Appleton Laboratory -- Scanned by iCritical. -------------- next part -------------- An HTML attachment was scrubbed... URL: From tarek.ziade at ingeniweb.com Fri Dec 12 13:39:40 2008 From: tarek.ziade at ingeniweb.com (Tarek Ziade) Date: Fri, 12 Dec 2008 13:39:40 +0100 Subject: [Distutils] problems with PyPI? In-Reply-To: <494257FF.7050407@simplistix.co.uk> References: <494257FF.7050407@simplistix.co.uk> Message-ID: 2008/12/12 Chris Withers : > Hi All, > > Is anyone else having problems with PyPI? > I tried mailing the catalog-sig, but my message hasn't even shown up in the > archives: > > http://mail.python.org/pipermail/catalog-sig/ > > Looks like ReST isn't rendering for package descriptions, I've recently > uploaded new versions of these: Your rest is broken. here's how you can test it before you upload a version (I would like PyPI to display a warning in this case, it's in my pile of suggestions) MacZiade:testfixtures-1.4.0 tarek$ python setup.py --long-description | rst2html.py > /dev/null :124: (WARNING/2) Title underline too short. 1.4.0 (12 Dec 2008) ------------------ :131: (WARNING/2) Title underline too short. 1.3.1 (10 Dec 2008) ------------------ :131: (WARNING/2) Title underline too short. 1.3.1 (10 Dec 2008) ------------------ > > http://pypi.python.org/pypi/testfixtures > http://pypi.python.org/pypi/mailinglogger > > Also, the search and front-page-newest-packages appear to be broken... > > Am I doing something wrong or is PyPI hosed? > > cheers, > > Chris > > -- > Simplistix - Content Management, Zope & Python Consulting > - http://www.simplistix.co.uk > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > -- Tarek Ziad? - Directeur Technique INGENIWEB (TM) - SAS 50000 Euros - RC B 438 725 632 Bureaux de la Colline - 1 rue Royale - B?timent D - 9?me ?tage 92210 Saint Cloud - France Phone : 01.78.15.24.00 / Fax : 01 46 02 44 04 http://www.ingeniweb.com - une soci?t? du groupe Alter Way From jim at zope.com Fri Dec 12 13:49:41 2008 From: jim at zope.com (Jim Fulton) Date: Fri, 12 Dec 2008 07:49:41 -0500 Subject: [Distutils] [zc.buildout] distutils "scripts" keyword In-Reply-To: References: Message-ID: On Dec 12, 2008, at 7:34 AM, Pascoe, S (Stephen) wrote: > I'm coming across a lot of packages that use the distutils "scripts" > keyword to identify scripts (rather than the console_scripts entry- > point). Unfortunately these don't work with buildout, although they > are installed with easy_install. > > I think this should be a solvable problem with a recipe that extends > zc.recipe.egg. It would be better to teach the machinery in buildout used by zc.recipe.egg to deal with scripts the same way easy_install does. I'll look into this. Jim -- Jim Fulton Zope Corporation From chris at simplistix.co.uk Fri Dec 12 15:40:20 2008 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 12 Dec 2008 14:40:20 +0000 Subject: [Distutils] problems with PyPI? In-Reply-To: References: <494257FF.7050407@simplistix.co.uk> Message-ID: <494277D4.2070803@simplistix.co.uk> Tarek Ziade wrote: > Your rest is broken. here's how you can test it before you upload a version Thanks, yes, that was it. Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk From jdh2358 at gmail.com Mon Dec 15 18:47:57 2008 From: jdh2358 at gmail.com (John Hunter) Date: Mon, 15 Dec 2008 11:47:57 -0600 Subject: [Distutils] symlinks in sourcetree sdist breaking easy_install? Message-ID: <88e473830812150947x53c54295o43e20c3c44369a2f@mail.gmail.com> We are experiencing a problem related to symbolic links in our source tree/sdist that appears to breaking easy_install (using setuptools-0.6c9). To produce the bug:: jdhunter at bic128:~> PYTHONPATH=~/dev/lib/python2.5/site-packages:/home/jdhunter/devez/lib/python2.5/site-packages ~/dev/bin/easy_install --prefix=~/devez matplotlib Searching for matplotlib Reading http://pypi.python.org/simple/matplotlib/ Reading http://matplotlib.sourceforge.net Reading https://sourceforge.net/project/showfiles.php?group_id=80706&package_id=278194 Reading https://sourceforge.net/project/showfiles.php?group_id=80706&package_id=82474 Reading http://sourceforge.net/project/showfiles.php?group_id=80706 Best match: matplotlib 0.98.5 Downloading http://downloads.sourceforge.net/matplotlib/matplotlib-0.98.5.tar.gz?modtime=1229034572&big_mirror=0 Processing matplotlib-0.98.5.tar.gz Running matplotlib-0.98.5/setup.py -q bdist_egg --dist-dir /tmp/easy_install-qW11xj/matplotlib-0.98.5/egg-dist-tmp-QFgBsw ..snip snip [Edit setup.cfg to suppress the above messages] ============================================================================ error: lib/matplotlib/mpl-data/matplotlib.conf.template: No such file or directory Exception exceptions.OSError: (2, 'No such file or directory', 'src/image.cpp') in > ignored Exception exceptions.OSError: (2, 'No such file or directory', 'src/path.cpp') in > ignored Exception exceptions.OSError: (2, 'No such file or directory', 'src/backend_gdk.c') in > ignored Exception exceptions.OSError: (2, 'No such file or directory', 'src/backend_agg.cpp') in > ignored jdhunter at bic128:~> The file which triggers the first error is matplotlib.conf.template, which exists in lib/matplotlib/mpl-data. There is a symlink in doc/mpl_data which points to lib/matplotlib/mpl-data. Eg in the mpl src tree:: jdhunter at bic128:mpl> ls -ld lib/matplotlib/mpl-data/ doc/mpl_data lrwxrwxrwx 1 jdhunter jdhunter 27 Jun 2 2008 doc/mpl_data -> ../lib/matplotlib/mpl-data/ drwxrwxr-x 6 jdhunter jdhunter 4096 Dec 10 13:09 lib/matplotlib/mpl-data/ When we create the sdist, here is the output matching that file:: jdhunter at bic128:mpl> python setup.py sdist > sdist.out jdhunter at bic128:mpl> grep matplotlib.conf.template sdist.out hard linking doc/mpl_data/matplotlib.conf.template -> matplotlib-0.98.5/doc/mpl_data hard linking lib/matplotlib/mpl-data/matplotlib.conf.template -> matplotlib-0.98.5/lib/matplotlib/mpl-data In the tarball, the link looks backwards to me: the tarball says lib/matplotlib/mpl-data/matplotlib.conf.template links to doc/mpl_data/matplotlib.conf.template and has zero content jdhunter at bic128:tmp> tar tvfz matplotlib-0.98.5.tar.gz |grep matplotlib.conf.template -rw-r--r-- cmoad/staff 13838 2008-12-09 17:53 matplotlib-0.98.5/doc/mpl_data/matplotlib.conf.templatehrw-r--r-- cmoad/staff 0 2008-12-09 17:53 matplotlib-0.98.5/lib/matplotlib/mpl-data/matplotlib.conf.template link to matplotlib-0.98.5/doc/mpl_data/matplotlib.conf.template If you unpack the tarball using a standard incantation, a copy is made:: jdhunter at bic128:tmp> tar xfz matplotlib-0.98.5.tar.gz jdhunter at bic128:tmp> cd matplotlib-0.98.5/ jdhunter at bic128:matplotlib-0.98.5> ls -ld doc/mpl_data/matplotlib.conf.template lib/matplotlib/mpl-data/matplotlib.conf.template -rw-r--r-- 2 jdhunter jdhunter 13838 Dec 9 17:53 doc/mpl_data/matplotlib.conf.template -rw-r--r-- 2 jdhunter jdhunter 13838 Dec 9 17:53 lib/matplotlib/mpl-data/matplotlib.conf.template I think (guess) what may be happening is that setuptools are reading the tarball using the python tarfile reader and the link is getting botched. However, I *can* read the "missing" file using tarfile:: In [17]: import tarfile In [18]: t = tarfile.open('dist/matplotlib-0.98.5.tar.gz') In [19]: o = t.extractfile('matplotlib-0.98.5/lib/matplotlib/mpl-data/matplotlib.conf') In [20]: s = o.read() In [21]: print s[:80] # MPLConfig - plaintext (in .conf format) # This is a sample matplotlib configu So I am not sure exactly what is going wrong but I think broken handling of the link is playing a role. I will probably work around this by removing the links and making hard copies myself before building the sdist and eggs, unless I hear something more intelligent from someone or come up with something better. From jdh2358 at gmail.com Mon Dec 15 21:40:08 2008 From: jdh2358 at gmail.com (John Hunter) Date: Mon, 15 Dec 2008 14:40:08 -0600 Subject: [Distutils] leading underscores in filenames Message-ID: <88e473830812151240k64731c77v9cf3431f92d16884@mail.gmail.com> I am seeing some strange behavior in files with leading underscores. I have some python extension code with names like:: src/_somefile.cpp and in my sdist they are being added in duplicate, one with the leading underscore removed. src/somefile.cpp src/_somefile.cpp This is causing easy_setup to complain at the end about not being able to remove certain files. jdhunter at bic128:mpl98.5> find . -name backend_agg.cpp jdhunter at bic128:mpl98.5> find . -name _backend_agg.cpp ./src/_backend_agg.cpp jdhunter at bic128:mpl98.5> python setup.py sdist > sdist.out jdhunter at bic128:mpl98.5> grep backend_agg.cpp sdist.out copying src/_backend_agg.cpp -> matplotlib-0.98.5.1/src copying src/backend_agg.cpp -> matplotlib-0.98.5.1/src When I run easy_install on the src tarball, I then get:: Installed /home/jdhunter/devez/lib/python2.5/site-packages/matplotlib-0.98.5.1-py2.5-linux-x86_64.egg Processing dependencies for matplotlib==0.98.5.1 Finished processing dependencies for matplotlib==0.98.5.1 Exception exceptions.OSError: (2, 'No such file or directory', 'src/image.cpp') in > ignored Exception exceptions.OSError: (2, 'No such file or directory', 'src/path.cpp') in > ignored Exception exceptions.OSError: (2, 'No such file or directory', 'src/backend_gdk.c') in > ignored Exception exceptions.OSError: (2, 'No such file or directory', 'src/backend_agg.cpp') in > ignored Using the standard distutils with python 2.5.1 From ianb at colorstudy.com Wed Dec 17 01:44:53 2008 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 16 Dec 2008 18:44:53 -0600 Subject: [Distutils] "Stable" versions Message-ID: <49484B85.50205@colorstudy.com> I've received a request that pip be able to be restricted to "stable" versions. It was suggested some kind of --alpha-ok or --beta-ok option, but this seems crude. I'd rather have it be part of the requirement. Maybe Package==stable? But there's also reason to do something like Package>=2.0,stable. So while pkg_resources can parse Package==stable easily enough, it's not by itself really expressive enough (I suppose Package>=2.0,==stable actually works okay). Also, there needs to be a definition of what versions are "stable". And maybe a distinction between beta/rc and development, though I'm less worried about that. Are there definitions of this? Does zc.buildout do this? -- Ian Bicking : ianb at colorstudy.com : http://blog.ianbicking.org From pje at telecommunity.com Wed Dec 17 02:19:17 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 16 Dec 2008 20:19:17 -0500 Subject: [Distutils] "Stable" versions In-Reply-To: <49484B85.50205@colorstudy.com> References: <49484B85.50205@colorstudy.com> Message-ID: <20081217011738.42ECC3A4119@sparrow.telecommunity.com> At 06:44 PM 12/16/2008 -0600, Ian Bicking wrote: >I've received a request that pip be able to be restricted to >"stable" versions. It was suggested some kind of --alpha-ok or >--beta-ok option, but this seems crude. I'd rather have it be part >of the requirement. Maybe Package==stable? But there's also reason >to do something like Package>=2.0,stable. So while pkg_resources >can parse Package==stable easily enough, it's not by itself really >expressive enough (I suppose Package>=2.0,==stable actually works okay). > >Also, there needs to be a definition of what versions are >"stable". And maybe a distinction between beta/rc and development, >though I'm less worried about that. Are there definitions of this? I would suggest an option that sets a minimum allowable alphanumeric code, e.g. "b" to require beta or better, "c" for release candidate, etc. It's easy to scan a parsed version for pre-release tags that match. Setting such an option to "final" would reject any version containing a pre-release tag. From ianb at colorstudy.com Wed Dec 17 02:29:26 2008 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 16 Dec 2008 19:29:26 -0600 Subject: [Distutils] "Stable" versions In-Reply-To: <20081217011738.42ECC3A4119@sparrow.telecommunity.com> References: <49484B85.50205@colorstudy.com> <20081217011738.42ECC3A4119@sparrow.telecommunity.com> Message-ID: <494855F6.2040301@colorstudy.com> Phillip J. Eby wrote: > At 06:44 PM 12/16/2008 -0600, Ian Bicking wrote: >> I've received a request that pip be able to be restricted to "stable" >> versions. It was suggested some kind of --alpha-ok or --beta-ok >> option, but this seems crude. I'd rather have it be part of the >> requirement. Maybe Package==stable? But there's also reason to do >> something like Package>=2.0,stable. So while pkg_resources can parse >> Package==stable easily enough, it's not by itself really expressive >> enough (I suppose Package>=2.0,==stable actually works okay). >> >> Also, there needs to be a definition of what versions are "stable". >> And maybe a distinction between beta/rc and development, though I'm >> less worried about that. Are there definitions of this? > > I would suggest an option that sets a minimum allowable alphanumeric > code, e.g. "b" to require beta or better, "c" for release candidate, > etc. It's easy to scan a parsed version for pre-release tags that > match. Setting such an option to "final" would reject any version > containing a pre-release tag. Well, the problem with an option is that it's global. I can imagine wanting to install FirstPackage==stable, and SomethingElse (no specific version requirement). -- Ian Bicking : ianb at colorstudy.com : http://blog.ianbicking.org From benji at zope.com Wed Dec 17 03:04:58 2008 From: benji at zope.com (Benji York) Date: Tue, 16 Dec 2008 21:04:58 -0500 Subject: [Distutils] "Stable" versions In-Reply-To: <49484B85.50205@colorstudy.com> References: <49484B85.50205@colorstudy.com> Message-ID: On Tue, Dec 16, 2008 at 7:44 PM, Ian Bicking wrote: > Also, there needs to be a definition of what versions are "stable". And > maybe a distinction between beta/rc and development, though I'm less worried > about that. Are there definitions of this? Does zc.buildout do this? http://pypi.python.org/pypi/zc.buildout#preferring-final-releases -- Benji York Senior Software Engineer Zope Corporation From ben+python at benfinney.id.au Wed Dec 17 06:16:16 2008 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 17 Dec 2008 16:16:16 +1100 Subject: [Distutils] "Stable" versions References: <49484B85.50205@colorstudy.com> Message-ID: <87y6yfe5zz.fsf@benfinney.id.au> Ian Bicking writes: > Also, there needs to be a definition of what versions are "stable". > And maybe a distinction between beta/rc and development, though I'm > less worried about that. Are there definitions of this? A further complication: If ?Foo? at its latest stable version has a dependency on a non-stable version of ?Bar? (however ?stable? ends up being defined), what happens when asking for ?only stable versions of Foo?? -- \ ?Even if the voices in my head are not real, they have pretty | `\ good ideas.? ?anonymous | _o__) | Ben Finney From felix.schwarz at web.de Wed Dec 17 18:15:09 2008 From: felix.schwarz at web.de (Felix Schwarz) Date: Wed, 17 Dec 2008 18:15:09 +0100 Subject: [Distutils] easy_install adds bad interpreter shebang to installed scripts Message-ID: <4949339D.7050801@web.de> Hi *, I'm using Python 2.5 on Fedora Linux 9. If the path to the interpreter contains spaces and I install some eggs with scripts using easy_install, all installed scripts will have a non-functional shebang line like this: #!"/home/foo bar/bin/python" On Linux (and I think this is true for all Un*x flavors) you can not quote the shebang path and there is no way around this [1]. To make it more clear where the problem is I attached some kind of 'patch'. My idea was that quoting shebang paths seem to work on Windows (at least this was my impression from reading the changelogs) but on Linux there is no way to generate a working shebang line with an absolute path. Therefore it would help me if /usr/bin/env is queried. Probably this will break for some people but currently the behavior is broken for everyone. With virtualenv you have to call activate (instead of calling the installed script directly) but then everything will work fine. Do you consider this a kind of bug you might be inclined to fix? fs [1] http://groups.google.com/group/gnu.bash.bug/browse_thread/thread/861a98e53be305c3 -------------- next part -------------- A non-text attachment was scrubbed... Name: spaces.patch Type: text/x-patch Size: 770 bytes Desc: not available URL: From pje at telecommunity.com Wed Dec 17 21:50:31 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 17 Dec 2008 15:50:31 -0500 Subject: [Distutils] easy_install adds bad interpreter shebang to installed scripts In-Reply-To: <4949339D.7050801@web.de> References: <4949339D.7050801@web.de> Message-ID: <20081217204850.F31913A40B2@sparrow.telecommunity.com> At 06:15 PM 12/17/2008 +0100, Felix Schwarz wrote: >Index: setuptools/command/easy_install.py >=================================================================== >--- setuptools/command/easy_install.py (Revision 67824) >+++ setuptools/command/easy_install.py (Arbeitskopie) >@@ -1418,8 +1418,12 @@ > if options: options = ' '+options > if wininst: > executable = "python.exe" >+ elif sys.platform=='win32': >+ executable = nt_quote_arg(executable) >+ elif ' ' in executable: >+ executable = '/usr/bin/env python' > else: >- executable = nt_quote_arg(executable) >+ executable = executable > hdr = "#!%(executable)s%(options)s\n" % locals() > if unicode(hdr,'ascii','ignore').encode('ascii') != hdr: > # Non-ascii path to sys.executable, use -x to prevent warnings I'm okay with the change in principle, but in practice, just dropping to "/usr/bin/env python" isn't acceptable. It should point env to the specific sys.executable, just like the "fix_jython_executable()" function does. From ianb at colorstudy.com Wed Dec 17 22:00:36 2008 From: ianb at colorstudy.com (Ian Bicking) Date: Wed, 17 Dec 2008 15:00:36 -0600 Subject: [Distutils] easy_install adds bad interpreter shebang to installed scripts In-Reply-To: <20081217204850.F31913A40B2@sparrow.telecommunity.com> References: <4949339D.7050801@web.de> <20081217204850.F31913A40B2@sparrow.telecommunity.com> Message-ID: <49496874.3010301@colorstudy.com> Phillip J. Eby wrote: > At 06:15 PM 12/17/2008 +0100, Felix Schwarz wrote: > >> Index: setuptools/command/easy_install.py >> =================================================================== >> --- setuptools/command/easy_install.py (Revision 67824) >> +++ setuptools/command/easy_install.py (Arbeitskopie) >> @@ -1418,8 +1418,12 @@ >> if options: options = ' '+options >> if wininst: >> executable = "python.exe" >> + elif sys.platform=='win32': >> + executable = nt_quote_arg(executable) >> + elif ' ' in executable: >> + executable = '/usr/bin/env python' >> else: >> - executable = nt_quote_arg(executable) >> + executable = executable >> hdr = "#!%(executable)s%(options)s\n" % locals() >> if unicode(hdr,'ascii','ignore').encode('ascii') != hdr: >> # Non-ascii path to sys.executable, use -x to prevent warnings > > I'm okay with the change in principle, but in practice, just dropping to > "/usr/bin/env python" isn't acceptable. It should point env to the > specific sys.executable, just like the "fix_jython_executable()" > function does. Does this work? #!/usr/bin/env "/path/to/weird path/python" -- Ian Bicking : ianb at colorstudy.com : http://blog.ianbicking.org From ianb at colorstudy.com Wed Dec 17 22:02:17 2008 From: ianb at colorstudy.com (Ian Bicking) Date: Wed, 17 Dec 2008 15:02:17 -0600 Subject: [Distutils] Unsetting something from distutils.cfg Message-ID: <494968D9.2030906@colorstudy.com> Is there any way, when you have a setting in distutils.cfg, to unset that value? For instance, if you've set: [install] prefix = something Is there a way to get distutils to ignore this setting? Setting it to the empty string doesn't seem to do it, and I don't know of any other way. -- Ian Bicking : ianb at colorstudy.com : http://blog.ianbicking.org From felix.schwarz at web.de Wed Dec 17 22:24:43 2008 From: felix.schwarz at web.de (Felix Schwarz) Date: Wed, 17 Dec 2008 22:24:43 +0100 Subject: [Distutils] easy_install adds bad interpreter shebang to installed scripts In-Reply-To: <20081217204850.F31913A40B2@sparrow.telecommunity.com> References: <4949339D.7050801@web.de> <20081217204850.F31913A40B2@sparrow.telecommunity.com> Message-ID: <49496E1B.2030102@web.de> Phillip J. Eby schrieb: > I'm okay with the change in principle, but in practice, just dropping to > "/usr/bin/env python" isn't acceptable. It should point env to the > specific sys.executable, just like the "fix_jython_executable()" > function does. Maybe I misunderstood you but I think this is not possible because the executable path may contain spaces. The shebang line will be split along spaces so arguments must not contain spaces (they will be considered as two separate arguments). fs From pje at telecommunity.com Wed Dec 17 22:29:54 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 17 Dec 2008 16:29:54 -0500 Subject: [Distutils] easy_install adds bad interpreter shebang to installed scripts In-Reply-To: <49496874.3010301@colorstudy.com> References: <4949339D.7050801@web.de> <20081217204850.F31913A40B2@sparrow.telecommunity.com> <49496874.3010301@colorstudy.com> Message-ID: <20081217212814.2D9053A4108@sparrow.telecommunity.com> At 03:00 PM 12/17/2008 -0600, Ian Bicking wrote: >Does this work? > >#!/usr/bin/env "/path/to/weird path/python" Depends on the Unix, unfortunately. Some would need it without the quotes to work, and some probably won't work with or without it. My point was more that it should at least try for something specific, i.e. python2.5, rather than just 'python', and perhaps verify that searching PATH for that string yields sys.executable. If not, then you already know it's not going to work right, and you might as well issue a warning right away. From felix.schwarz at schwarz.eu Wed Dec 17 22:17:07 2008 From: felix.schwarz at schwarz.eu (Felix Schwarz) Date: Wed, 17 Dec 2008 22:17:07 +0100 Subject: [Distutils] easy_install adds bad interpreter shebang to installed scripts In-Reply-To: <49496874.3010301@colorstudy.com> References: <4949339D.7050801@web.de> <20081217204850.F31913A40B2@sparrow.telecommunity.com> <49496874.3010301@colorstudy.com> Message-ID: <49496C53.70306@schwarz.eu> Ian Bicking schrieb: > Does this work? > > #!/usr/bin/env "/path/to/weird path/python" No (at least not for me). As far as I understand this, no kind of quoting or escaping will work on whole shebang line because the kernel just splits the string and passes them as arguments to the application. No shell is involved there. fs From pje at telecommunity.com Wed Dec 17 23:29:12 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 17 Dec 2008 17:29:12 -0500 Subject: [Distutils] easy_install adds bad interpreter shebang to installed scripts In-Reply-To: <49496C53.70306@schwarz.eu> References: <4949339D.7050801@web.de> <20081217204850.F31913A40B2@sparrow.telecommunity.com> <49496874.3010301@colorstudy.com> <49496C53.70306@schwarz.eu> Message-ID: <20081217222732.174983A4108@sparrow.telecommunity.com> At 10:17 PM 12/17/2008 +0100, Felix Schwarz wrote: >Ian Bicking schrieb: >>Does this work? >>#!/usr/bin/env "/path/to/weird path/python" > >No (at least not for me). Did you actually try that, as opposed to the version you showed before? There *is* a difference. >As far as I understand this, no kind of quoting or escaping will >work on whole shebang line because the kernel just splits the string >and passes them as arguments to the application. You understand incorrectly; shebang parsing is platform dependent. See: http://www.in-ulm.de/~mascheck/various/shebang/ In particular, I just tested Linux (CentOS 4.3, to be precise) and found that this works: #!/bin/env /path/witha/s pace/init However, if I understand correctly, this would not work correctly on certain *BSD-derived operating systems, which would pass "/path/witha/s" and "pace/init" to env as separate arguments, instead of a single combined argument, thereby keeping it from working. It seems that OS X would be a good place to test also, especially since IIRC it is *BSD-derived. From tseaver at palladion.com Thu Dec 18 00:48:56 2008 From: tseaver at palladion.com (Tres Seaver) Date: Wed, 17 Dec 2008 18:48:56 -0500 Subject: [Distutils] easy_install adds bad interpreter shebang to installed scripts In-Reply-To: <4949339D.7050801@web.de> References: <4949339D.7050801@web.de> Message-ID: <49498FE8.2050805@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Felix Schwarz wrote: > I'm using Python 2.5 on Fedora Linux 9. If the path to the interpreter > contains spaces and I install some eggs with scripts using easy_install, > all installed scripts will have a non-functional shebang line like this: > #!"/home/foo bar/bin/python" > > On Linux (and I think this is true for all Un*x flavors) you can not > quote the shebang path and there is no way around this [1]. > > To make it more clear where the problem is I attached some kind of > 'patch'. My idea was that quoting shebang paths seem to work on Windows > (at least this was my impression from reading the changelogs) but on > Linux there is no way to generate a working shebang line with an > absolute path. Therefore it would help me if /usr/bin/env is queried. > > Probably this will break for some people but currently the behavior is > broken for everyone. With virtualenv you have to call activate (instead > of calling the installed script directly) but then everything will work > fine. > > Do you consider this a kind of bug you might be inclined to fix? I *never* want to see /usr/bin/env in a generated script's shebang: I want the script to use the python which was used to install it, and no other./ - From my POV, I don't understand why on a Linux box you would choose to ape Windows and use broken paths to important system components: "just say no" to paths with spaces works for me. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJSY/o+gerLs4ltQ4RAotoAJ9AQQPiWbu4B4wE8/206AAa+inCUQCfaJ1i vN1giPJfE6M2vCBlYtZANTQ= =/Y9F -----END PGP SIGNATURE----- From tarek.ziade at ingeniweb.com Thu Dec 18 10:40:20 2008 From: tarek.ziade at ingeniweb.com (Tarek Ziade) Date: Thu, 18 Dec 2008 10:40:20 +0100 Subject: [Distutils] Unsetting something from distutils.cfg In-Reply-To: <494968D9.2030906@colorstudy.com> References: <494968D9.2030906@colorstudy.com> Message-ID: I don't think you can unset a value. You can override it from the command line though $ python setup.py install --prefix=xxx But if you need to ignore the "prefix" option, I think you need to write your own install command to override this behavior. 2008/12/17 Ian Bicking : > Is there any way, when you have a setting in distutils.cfg, to unset that > value? For instance, if you've set: > > [install] > prefix = something > > Is there a way to get distutils to ignore this setting? Setting it to the > empty string doesn't seem to do it, and I don't know of any other way. > > -- > Ian Bicking : ianb at colorstudy.com : http://blog.ianbicking.org > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > -- Tarek Ziad? - Directeur Technique INGENIWEB (TM) - SAS 50000 Euros - RC B 438 725 632 Bureaux de la Colline - 1 rue Royale - B?timent D - 9?me ?tage 92210 Saint Cloud - France Phone : 01.78.15.24.00 / Fax : 01 46 02 44 04 http://www.ingeniweb.com - une soci?t? du groupe Alter Way From felix.schwarz at web.de Thu Dec 18 16:20:28 2008 From: felix.schwarz at web.de (Felix Schwarz) Date: Thu, 18 Dec 2008 16:20:28 +0100 Subject: [Distutils] easy_install adds bad interpreter shebang to installed scripts In-Reply-To: <20081217222732.174983A4108@sparrow.telecommunity.com> References: <4949339D.7050801@web.de> <20081217204850.F31913A40B2@sparrow.telecommunity.com> <49496874.3010301@colorstudy.com> <49496C53.70306@schwarz.eu> <20081217222732.174983A4108@sparrow.telecommunity.com> Message-ID: <494A6A3C.3040807@web.de> Phillip J. Eby schrieb: > At 10:17 PM 12/17/2008 +0100, Felix Schwarz wrote: >> Ian Bicking schrieb: >>> Does this work? >>> #!/usr/bin/env "/path/to/weird path/python" >> >> No (at least not for me). > > Did you actually try that, as opposed to the version you showed before? > There *is* a difference. Of course I tried. :-) I just tried again and it still does not work. However I noticed that it works without any quoting for me too. Which I thought I tried before... So for me (using Fedora Linux with Python 2.5, nothing unusual) the solution would be to add /usr/bin/env Probably this is the way to go forward. Let's see if the behavior is different for BSDs (I have neither a Mac nor any BSD-based system available right now). fs From exarkun at divmod.com Thu Dec 18 16:40:03 2008 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Thu, 18 Dec 2008 10:40:03 -0500 Subject: [Distutils] easy_install adds bad interpreter shebang to installed scripts In-Reply-To: <494A6A3C.3040807@web.de> Message-ID: <20081218154003.20272.1569386302.divmod.quotient.22613@ohm> On Thu, 18 Dec 2008 16:20:28 +0100, Felix Schwarz wrote: >Phillip J. Eby schrieb: >>At 10:17 PM 12/17/2008 +0100, Felix Schwarz wrote: >>>Ian Bicking schrieb: >>>>Does this work? >>>>#!/usr/bin/env "/path/to/weird path/python" >>> >>>No (at least not for me). >> >>Did you actually try that, as opposed to the version you showed before? >>There *is* a difference. > >Of course I tried. :-) >I just tried again and it still does not work. However I noticed that it >works without any quoting for me too. Which I thought I tried before... > >So for me (using Fedora Linux with Python 2.5, nothing unusual) the solution >would be to add >/usr/bin/env > >Probably this is the way to go forward. Let's see if the behavior is >different for BSDs (I have neither a Mac nor any BSD-based system available >right now). OS X results (note, ~/foo exists and is a directory): neutron:~ exarkun$ cat > foobar.sh #!/Users/exarkun/foo bar/python print 'yes' neutron:~ exarkun$ chmod u+x foobar.sh neutron:~ exarkun$ ./foobar.sh -bash: ./foobar.sh: /Users/exarkun/foo: bad interpreter: Permission denied neutron:~ exarkun$ cat > foobar.sh #!/Users/exarkun/foo\ bar/python print 'yes' neutron:~ exarkun$ ./foobar.sh -bash: ./foobar.sh: /Users/exarkun/foo\: bad interpreter: No such file or directory neutron:~ exarkun$ cat > foobar.sh #!"/Users/exarkun/foo bar/python" print "yes" neutron:~ exarkun$ ./foobar.sh -bash: ./foobar.sh: "/Users/exarkun/foo: bad interpreter: No such file or directory neutron:~ exarkun$ cat > foobar.sh #!'/Users/exarkun/foo bar/python' print 'yes' neutron:~ exarkun$ ./foobar.sh -bash: ./foobar.sh: '/Users/exarkun/foo: bad interpreter: No such file or directory neutron:~ exarkun$ cat > foobar.sh #!/usr/bin/env /Users/exarkun/foo bar/python print 'yes' neutron:~ exarkun$ ./foobar.sh env: /Users/exarkun/foo: Permission denied neutron:~ exarkun$ cat > foobar.sh #!/usr/bin/env /Users/exarkun/foo\ bar/python print 'yes' neutron:~ exarkun$ ./foobar.sh env: /Users/exarkun/foo\: No such file or directory neutron:~ exarkun$ cat > foobar.sh #!/usr/bin/env "/Users/exarkun/foo bar/python" print 'yes' neutron:~ exarkun$ ./foobar.sh env: "/Users/exarkun/foo: No such file or directory neutron:~ exarkun$ cat > foobar.sh #!/usr/bin/env '/Users/exarkun/foo bar/python' print 'yes' neutron:~ exarkun$ ./foobar.sh env: '/Users/exarkun/foo: No such file or directory neutron:~ exarkun$ Jean-Paul From pje at telecommunity.com Thu Dec 18 18:23:43 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 18 Dec 2008 12:23:43 -0500 Subject: [Distutils] easy_install adds bad interpreter shebang to installed scripts In-Reply-To: <20081218154003.20272.1569386302.divmod.quotient.22613@ohm> References: <494A6A3C.3040807@web.de> <20081218154003.20272.1569386302.divmod.quotient.22613@ohm> Message-ID: <20081218172351.EC5D13A4120@sparrow.telecommunity.com> At 10:40 AM 12/18/2008 -0500, Jean-Paul Calderone wrote: >On Thu, 18 Dec 2008 16:20:28 +0100, Felix Schwarz > wrote: >>Phillip J. Eby schrieb: >>>At 10:17 PM 12/17/2008 +0100, Felix Schwarz wrote: >>>>Ian Bicking schrieb: >>>>>Does this work? >>>>>#!/usr/bin/env "/path/to/weird path/python" >>>> >>>>No (at least not for me). >>> >>>Did you actually try that, as opposed to the version you showed >>>before? There *is* a difference. >> >>Of course I tried. :-) >>I just tried again and it still does not work. However I noticed >>that it works without any quoting for me too. Which I thought I tried before... >> >>So for me (using Fedora Linux with Python 2.5, nothing unusual) the >>solution would be to add >>/usr/bin/env >> >>Probably this is the way to go forward. Let's see if the behavior >>is different for BSDs (I have neither a Mac nor any BSD-based >>system available right now). > >OS X results (note, ~/foo exists and is a directory): Hm. So no matter what, it's not going to work on OS X; I guess that means we could just go with the way that works on Linux. ;-) It appears that OS X is doing what I'd expect a *BSD to - it's splitting the #! line on spaces and feeding each one to env as a separate argument -- quotes and all. The only other way to do this that I can think of would be to write the file with a bilingual shell script header, such that it does an exec of the appropriate Python on $0. That would work on all platforms, at the cost of incurring an extra exec -- but then, /bin/env does that too. From jaraco at jaraco.com Thu Dec 18 23:41:07 2008 From: jaraco at jaraco.com (Jason R. Coombs) Date: Thu, 18 Dec 2008 15:41:07 -0700 Subject: [Distutils] Error downloading pythonutils from pypi: Can't process plain .py files Message-ID: <750B64C66078B34D918257A1AC004DB20546EE@messiah.jaraco.com> Downloading pythonutils from pypi fails even though it is downloading a perfectly good .zip source file. > easy_install pythonutils==0.3.0 Searching for pythonutils==0.3.0 Reading http://pypi.python.org/simple/pythonutils/ Couldn't find index page for 'pythonutils' (maybe misspelled?) Scanning index of all packages (this may take a while) Reading http://pypi.python.org/simple/ Reading http://pypi.python.org/simple/Pythonutils/ Reading http://www.voidspace.org.uk/python/modules.shtml#pythonutils Reading http://www.voidspace.org.uk/cgi-bin/voidspace/downman.py?file=pythonutils-0. 2.3.zip Reading http://www.voidspace.org.uk/cgi-bin/voidspace/downman.py?file=pythonutils-0. 2.5.zip Reading http://www.voidspace.org.uk/cgi-bin/voidspace/downman.py?file=pythonutils-0. 3.0.zip No local packages or download links found for pythonutils==0.3.0 error: Could not find suitable distribution for Requirement.parse('pythonutils==0.3.0') If I download the .zip file manually and easy_install the zip file, the install goes great. I think the problem is related to the .py name appearing in the filename. > easy_install http://www.voidspace.org.uk/cgi-bin/voidspace/downman.py?file=pythonutils-0. 3.0.zip Downloading http://www.voidspace.org.uk/cgi-bin/voidspace/downman.py?file=pythonutils-0. 3.0.zip error: Can't process plain .py files without an '#egg=name-version' suffix to enable automatic setup script generation. I'm guessing that whatever easy_install uses for downloading the file ends up naming it downman.py (or something similar). I checked the headers on the response from voidspace, and I see voidspace is in fact supplying the content-disposition header: content-disposition: attachment; filename= "pythonutils-0.3.0.zip" So it appears that whatever downloader easy_install is using isn't honoring the content-disposition header. or the check for python-ness is otherwise getting a false positive. This problem was observed in Windows Vista 64-bit with Python 2.5.2 32-bit and setuptools 0.6c9. Should I post a bug in bugtracker on this issue? Regards, Jason -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 6998 bytes Desc: not available URL: From pje at telecommunity.com Fri Dec 19 03:58:49 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 18 Dec 2008 21:58:49 -0500 Subject: [Distutils] Error downloading pythonutils from pypi: Can't process plain .py files In-Reply-To: <750B64C66078B34D918257A1AC004DB20546EE@messiah.jaraco.com> References: <750B64C66078B34D918257A1AC004DB20546EE@messiah.jaraco.com> Message-ID: <20081219025713.5E72C3A411A@sparrow.telecommunity.com> At 03:41 PM 12/18/2008 -0700, Jason R. Coombs wrote: >Should I post a bug in bugtracker on this issue? No. easy_install requires that a URL path ends with the target filename. It *must* be able to determine what it's downloading BEFORE it actually downloads it, since it would otherwise have to download every possible file for a package in order to determine which one was a suitable version, python version, platform, etc. And it's not reasonable to expect it to grow special cases for every site that wants to hide the filename in the query string attached to a CGI script. If someone needs to have their files processed by a script, they could always make the path something like: downman.py/pythonutils-0.2.3.zip Using either PATH_INFO or mod_rewrite or something similar, at which point easy_install can tell it's looking at a source zipfile, rather than a .py script. From tseaver at palladion.com Fri Dec 19 04:58:31 2008 From: tseaver at palladion.com (Tres Seaver) Date: Thu, 18 Dec 2008 22:58:31 -0500 Subject: [Distutils] easy_install adds bad interpreter shebang to installed scripts In-Reply-To: <20081218172351.EC5D13A4120@sparrow.telecommunity.com> References: <494A6A3C.3040807@web.de> <20081218154003.20272.1569386302.divmod.quotient.22613@ohm> <20081218172351.EC5D13A4120@sparrow.telecommunity.com> Message-ID: <494B1BE7.3060900@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Phillip J. Eby wrote: > At 10:40 AM 12/18/2008 -0500, Jean-Paul Calderone wrote: >> On Thu, 18 Dec 2008 16:20:28 +0100, Felix Schwarz >> wrote: >>> Phillip J. Eby schrieb: >>>> At 10:17 PM 12/17/2008 +0100, Felix Schwarz wrote: >>>>> Ian Bicking schrieb: >>>>>> Does this work? >>>>>> #!/usr/bin/env "/path/to/weird path/python" >>>>> No (at least not for me). >>>> Did you actually try that, as opposed to the version you showed >>>> before? There *is* a difference. >>> Of course I tried. :-) >>> I just tried again and it still does not work. However I noticed >>> that it works without any quoting for me too. Which I thought I tried before... >>> >>> So for me (using Fedora Linux with Python 2.5, nothing unusual) the >>> solution would be to add >>> /usr/bin/env >>> >>> Probably this is the way to go forward. Let's see if the behavior >>> is different for BSDs (I have neither a Mac nor any BSD-based >>> system available right now). >> OS X results (note, ~/foo exists and is a directory): > > Hm. So no matter what, it's not going to work on OS X; I guess that > means we could just go with the way that works on Linux. ;-) > > It appears that OS X is doing what I'd expect a *BSD to - it's > splitting the #! line on spaces and feeding each one to env as a > separate argument -- quotes and all. > > The only other way to do this that I can think of would be to write > the file with a bilingual shell script header, such that it does an > exec of the appropriate Python on $0. That would work on all > platforms, at the cost of incurring an extra exec -- but then, > /bin/env does that too. I would way rather see that kind of solution than using 'env': scripts installed by easy_install should *not* use whatever python happens to be found at the moment on PATH. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJSxvn+gerLs4ltQ4RAoGpAJ41Xur84yvoRyVjmOxI041xqp1lgQCeK+qG ODdMvlZ2S2SUWIvMUOlag00= =ZgH0 -----END PGP SIGNATURE----- From ianb at colorstudy.com Fri Dec 19 08:33:55 2008 From: ianb at colorstudy.com (Ian Bicking) Date: Fri, 19 Dec 2008 01:33:55 -0600 Subject: [Distutils] easy_install adds bad interpreter shebang to installed scripts In-Reply-To: <494B1BE7.3060900@palladion.com> References: <494A6A3C.3040807@web.de> <20081218154003.20272.1569386302.divmod.quotient.22613@ohm> <20081218172351.EC5D13A4120@sparrow.telecommunity.com> <494B1BE7.3060900@palladion.com> Message-ID: <494B4E63.7060304@colorstudy.com> Tres Seaver wrote: > I would way rather see that kind of solution than using 'env': scripts > installed by easy_install should *not* use whatever python happens to be > found at the moment on PATH. I agree that's necessary, but I don't think anyone has been proposing that (well, except the initial proposal). I'm guessing the script in this case could look like: #!/bin/sh exec "path/to/python" -c "everything that would normally be in the body of the script" -- Ian Bicking : ianb at colorstudy.com : http://blog.ianbicking.org From jaraco at jaraco.com Fri Dec 19 15:41:42 2008 From: jaraco at jaraco.com (Jason R. Coombs) Date: Fri, 19 Dec 2008 07:41:42 -0700 Subject: [Distutils] Error downloading pythonutils from pypi: Can't process plain .py files In-Reply-To: <20081219025713.5E72C3A411A@sparrow.telecommunity.com> References: <750B64C66078B34D918257A1AC004DB20546EE@messiah.jaraco.com> <20081219025713.5E72C3A411A@sparrow.telecommunity.com> Message-ID: <750B64C66078B34D918257A1AC004DB20546F1@messiah.jaraco.com> Phillip, Thanks for your response. > -----Original Message----- > From: Phillip J. Eby [mailto:pje at telecommunity.com] > Sent: Thursday, 18 December, 2008 21:59 > > ... And > it's not reasonable to expect it to grow special cases for every site > that wants to hide the filename in the query string attached to a CGI > script. I absolutely agree it's not reasonable to grow special cases... but the RFCs do provide a standard for describing the content disposition. Perhaps it would be worthwhile to handle that general case. Specifically, one could request the headers for a given URL to determine if a content-disposition filename is supplied. I've put together some sample code which is short, robust, and clean (I hope). This code demonstrates how one might go about querying the content-disposition. See it at: http://paste.turbogears.org/paste/19813 With that code, the filename_test code could go something like this: filename = get_content_disposition_filename(url) or get_filename_the_old_way(url) test_valid_egg_distro(filename) ... or alternatively, one could only perform the content-disposition retrieval when the filename _appears_ to be invalid. > > If someone needs to have their files processed by a script, they > could always make the path something like: > > downman.py/pythonutils-0.2.3.zip > > Using either PATH_INFO or mod_rewrite or something similar, at which > point easy_install can tell it's looking at a source zipfile, rather > than a .py script. While I agree it's unreasonable for setuptools to have to grow to handle nonstandard cases, I would say the same argument applies to the content providers. They shouldn't have to alter their deployment architecture to accommodate limitations in setuptools, particularly when they're following the web standards. The above solution handles the general case where files can be served through many different content management systems and not just as static file references in the URL. This allows for special urls like http://www.mycompany.com/myproject/latest_distro.jsp, which might generate the content dynamically. Note that the voidspace author is not attempting to "hide" the filename in the get parameters. He's probably using a sort of content management system. Would you consider a patch that incorporates this more robust behavior? Regards, Jason -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 6998 bytes Desc: not available URL: From pje at telecommunity.com Sat Dec 20 02:02:53 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 19 Dec 2008 20:02:53 -0500 Subject: [Distutils] Error downloading pythonutils from pypi: Can't process plain .py files In-Reply-To: <750B64C66078B34D918257A1AC004DB20546F1@messiah.jaraco.com> References: <750B64C66078B34D918257A1AC004DB20546EE@messiah.jaraco.com> <20081219025713.5E72C3A411A@sparrow.telecommunity.com> <750B64C66078B34D918257A1AC004DB20546F1@messiah.jaraco.com> Message-ID: <20081220010112.C89953A40A7@sparrow.telecommunity.com> At 07:41 AM 12/19/2008 -0700, Jason R. Coombs wrote: >Phillip, > Thanks for your response. > > > -----Original Message----- > > From: Phillip J. Eby [mailto:pje at telecommunity.com] > > Sent: Thursday, 18 December, 2008 21:59 > > > > ... And > > it's not reasonable to expect it to grow special cases for every site > > that wants to hide the filename in the query string attached to a CGI > > script. > >I absolutely agree it's not reasonable to grow special cases... but the RFCs >do provide a standard for describing the content disposition. Perhaps it >would be worthwhile to handle that general case. Specifically, one could >request the headers for a given URL to determine if a content-disposition >filename is supplied. Please re-read my original message; easy_install can't retrieve every URL in order to determine which URL to retrieve in the first place. >... or alternatively, one could only perform the content-disposition >retrieval when the filename _appears_ to be invalid. ...which would mean it would then have to follow every NON-download link on the page. >Would you consider a patch that incorporates this more robust behavior? If you can come up with a way to do it without retrieving any non-download URLs, and that doesn't need a special case for every kind of CMS, yes. I just don't think that those requirements can be met. Now, for the specific case of the apparent filename not matching the actual filename, it's possible you could do something about that. But it would only work for voidspace and anybody else using a .py file as their wrapping script, because the only reason easy_install's even *considering* that URL a download candidate is the .py extension. Without it, it wouldn't be considered a download URL to start with, let alone actually retrieved. (So, it won't address Java, PHP, or Zope CMS programs without more special cases.) From pje at telecommunity.com Sat Dec 20 02:16:03 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 19 Dec 2008 20:16:03 -0500 Subject: [Distutils] easy_install adds bad interpreter shebang to installed scripts In-Reply-To: <494B4E63.7060304@colorstudy.com> References: <494A6A3C.3040807@web.de> <20081218154003.20272.1569386302.divmod.quotient.22613@ohm> <20081218172351.EC5D13A4120@sparrow.telecommunity.com> <494B1BE7.3060900@palladion.com> <494B4E63.7060304@colorstudy.com> Message-ID: <20081220011553.0CEBC3A40F7@sparrow.telecommunity.com> At 01:33 AM 12/19/2008 -0600, Ian Bicking wrote: >Tres Seaver wrote: >>I would way rather see that kind of solution than using 'env': scripts >>installed by easy_install should *not* use whatever python happens to be >>found at the moment on PATH. > >I agree that's necessary, but I don't think anyone has been >proposing that (well, except the initial proposal). I'm guessing >the script in this case could look like: > >#!/bin/sh >exec "path/to/python" -c "everything that would normally be in the >body of the script" Actually, I was thinking more like: #!/bin/sh # standard easy_install comment lines here """:" exec '/path/to/python' "$0" "$@" """ # rest of the code here The idea being to preserve the inspectable info that easy_install puts in the first few lines of script comments. And of course, to properly pass through sys.argv, etc. The path to sys.executable also needs to be properly escaped by the code writing the script. From devel at jonasmelian.com Mon Dec 22 10:33:40 2008 From: devel at jonasmelian.com (devel at jonasmelian.com) Date: Mon, 22 Dec 2008 09:33:40 +0000 Subject: [Distutils] Compiling with Pyrex Message-ID: <1229938420.4350.1291286233@webmail.messagingengine.com> When I use the next command in my home system: $ python setup.py develop Pyrex compiles the '.pyx' file without any problem. But after of uploading it to Pypi, and when is installed via 'easy_install' it doesn't builds any more. (I had to upload the '.c' file compiled on my system) You can see here how has been configured: http://www.bitbucket.org/ares/bcryptwrap/src/tip/setup.py and here where I had to add any files: http://www.bitbucket.org/ares/bcryptwrap/src/tip/MANIFEST.in Note that in the last verion in Pypi I've only one of the next sentences but I had both ones in anterior revisions. setup_requires='Pyrex', install_requires='Pyrex', It download and install Pyrex before of my package, but Pyrex doesn't compile the file. From ziade.tarek at gmail.com Mon Dec 22 11:49:12 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Mon, 22 Dec 2008 11:49:12 +0100 Subject: [Distutils] Compiling with Pyrex In-Reply-To: <1229938420.4350.1291286233@webmail.messagingengine.com> References: <1229938420.4350.1291286233@webmail.messagingengine.com> Message-ID: <94bdd2610812220249k64931f81n98d462dcdcf75814@mail.gmail.com> Hello develop works with your environment, so it might not behave like install would. To make your package installable, I believe you have to hook a build_ext step in your setup.py, so Cython (Pyrex) can use your pyx file; I don't know how this can be done with older versions, but Cython provides a "build_pyx'" command that is used for example in lxml see http://codespeak.net/svn/lxml/trunk/setupinfo.py Good luck On Mon, Dec 22, 2008 at 10:33 AM, wrote: > When I use the next command in my home system: > > $ python setup.py develop > > Pyrex compiles the '.pyx' file without any problem. But after of > uploading it to Pypi, and when is installed via 'easy_install' it > doesn't builds any more. (I had to upload the '.c' file compiled on my > system) > > You can see here how has been configured: > http://www.bitbucket.org/ares/bcryptwrap/src/tip/setup.py > > and here where I had to add any files: > http://www.bitbucket.org/ares/bcryptwrap/src/tip/MANIFEST.in > > > Note that in the last verion in Pypi I've only one of the next sentences > but I had both ones in anterior revisions. > > setup_requires='Pyrex', > install_requires='Pyrex', > > It download and install Pyrex before of my package, but Pyrex doesn't > compile the file. > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ From pje at telecommunity.com Mon Dec 22 16:13:45 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 22 Dec 2008 10:13:45 -0500 Subject: [Distutils] Compiling with Pyrex In-Reply-To: <1229938420.4350.1291286233@webmail.messagingengine.com> References: <1229938420.4350.1291286233@webmail.messagingengine.com> Message-ID: <20081222151202.5FE6A3A40A0@sparrow.telecommunity.com> At 09:33 AM 12/22/2008 +0000, devel at jonasmelian.com wrote: >When I use the next command in my home system: > > $ python setup.py develop > >Pyrex compiles the '.pyx' file without any problem. But after of >uploading it to Pypi, and when is installed via 'easy_install' it >doesn't builds any more. (I had to upload the '.c' file compiled on my >system) > >You can see here how has been configured: >http://www.bitbucket.org/ares/bcryptwrap/src/tip/setup.py > >and here where I had to add any files: >http://www.bitbucket.org/ares/bcryptwrap/src/tip/MANIFEST.in > > >Note that in the last verion in Pypi I've only one of the next sentences >but I had both ones in anterior revisions. > > setup_requires='Pyrex', > install_requires='Pyrex', > >It download and install Pyrex before of my package, but Pyrex doesn't >compile the file. The setup_requires is the only one required; however, setuptools is checking for Pyrex's presence *before* it's available on sys.path. I don't know of any way to work around this, except to include pre-built .c files in your distribution. From rocky.bernstein at gmail.com Tue Dec 23 04:51:23 2008 From: rocky.bernstein at gmail.com (Rocky Bernstein) Date: Mon, 22 Dec 2008 22:51:23 -0500 Subject: [Distutils] Bug? zimpimporter putting in a misleading co_filename? Message-ID: <6cd6de210812221951w3d2581actf49d17305608ea13@mail.gmail.com> I recently cut over to using distutils to distribute open-source software. distutils decided to create an zipimporter egg. (Well, I'm sure it did what it did for good reason, I just am not all that aware of what I did to cause it too choose that.) When I inspect functions in that egg they seem to have a path that doesn't really exist. Is there something I did to create this improperly or perhaps this just they things are? For concreteness, select one of the eggs at http://code.google.com/p/pytracer and install one of these. I did this on GNU/Linux and Python 2.5 and I look at the co_filename of one of the methods: >>> import tracer >>> tracer.__dict__['size'].func_code.co_filename 'build/bdist.linux-i686/egg/tracer.py' But there is no file called "build/bdist.linux-686/egg/tracer.py" in the filesystem. Instead there is a member "tracer.py" inside /usr/lib/python2.5/site-packages/tracer-0.1.0-py2.5.egg'. Is there something I could to do cause this to happen? Related is the question is how one can determine the file location of an egg and the member for a given import. It looks to me that there is no standard way defined to do this across loaders that work with some sort of single archive like a single zipped egg. Is this correct? -------------- next part -------------- An HTML attachment was scrubbed... URL: From zooko at zooko.com Tue Dec 23 15:14:25 2008 From: zooko at zooko.com (zooko) Date: Tue, 23 Dec 2008 07:14:25 -0700 Subject: [Distutils] Bug? zimpimporter putting in a misleading co_filename? In-Reply-To: <6cd6de210812221951w3d2581actf49d17305608ea13@mail.gmail.com> References: <6cd6de210812221951w3d2581actf49d17305608ea13@mail.gmail.com> Message-ID: On Dec 22, 2008, at 20:51 PM, Rocky Bernstein wrote: > I recently cut over to using distutils to distribute open-source > software. distutils decided to create an zipimporter egg. (Well, > I'm sure it did what it did for good reason, I just am not all that > aware of what I did to cause it too choose that.) When I inspect > functions in that egg they seem to have a path that doesn't really > exist. Is there something I did to create this improperly or > perhaps this just they things are? This is one of the parts of issue33: http://bugs.python.org/setuptools/issue33 I just added the following comment to that issue ticket: Suggested workaround for people finding this ticket from a search engine query such as "distutils zip problem": 1. Put the following in your distutils config file: {{{ [easy_install] zip_ok=False }}} Then all packages that you easy_install will be unzipped. 2. When creating a package for others, pass {{{zip_ok=False}}} to {{{setup()}}}. This is a bit nasty, because that flag is really supposed to indicate whether there is something in the package that can't work from zip, and if you set that flag then this abrogrates the user's preference when installing -- there is no way for them to tell easy_install to install the package zipped in spite of the {{{zip_ok=False}}} flag. However, I have seen so many reports of problems with zip files, and zero reports of problems with unzipped files, and zero requests for zipped files, so I'm personally comfortable setting {{{zip_ok=False}}} just to help the user avoid the problems. From rocky.bernstein at gmail.com Tue Dec 23 16:11:00 2008 From: rocky.bernstein at gmail.com (Rocky Bernstein) Date: Tue, 23 Dec 2008 10:11:00 -0500 Subject: [Distutils] Bug? zimpimporter putting in a misleading co_filename? In-Reply-To: References: <6cd6de210812221951w3d2581actf49d17305608ea13@mail.gmail.com> Message-ID: <6cd6de210812230711p6fa7c774q3e59f09a3e954ede@mail.gmail.com> This is useful information and thanks for suggesting the workaround. However this does not address the issue which is not about how I can create an unzipped egg for my use or the use of others. (Or what I did or didn't do to create such an zipped egg). I am interested in writing tools that work with both zipped an unzipped eggs. To the extent that setuptools/distutils is responsible for setting the co_filename field in the code object what should go inside that field? And how can a programmer reliably and, if possible, uniformly untangle this to get/report a location. On Tue, Dec 23, 2008 at 9:14 AM, zooko wrote: > On Dec 22, 2008, at 20:51 PM, Rocky Bernstein wrote: > > I recently cut over to using distutils to distribute open-source software. >> distutils decided to create an zipimporter egg. (Well, I'm sure it did what >> it did for good reason, I just am not all that aware of what I did to cause >> it too choose that.) When I inspect functions in that egg they seem to have >> a path that doesn't really exist. Is there something I did to create this >> improperly or perhaps this just they things are? >> > > This is one of the parts of issue33: > > http://bugs.python.org/setuptools/issue33 > > I just added the following comment to that issue ticket: > > Suggested workaround for people finding this ticket from a search engine > query > such as "distutils zip problem": > > 1. Put the following in your distutils config file: > > {{{ > [easy_install] > zip_ok=False > }}} > > Then all packages that you easy_install will be unzipped. > > 2. When creating a package for others, pass {{{zip_ok=False}}} to > {{{setup()}}}. This is a bit nasty, because that flag is really supposed > to > indicate whether there is something in the package that can't work from > zip, and > if you set that flag then this abrogrates the user's preference when > installing > -- there is no way for them to tell easy_install to install the package > zippedin spite of the {{{zip_ok=False}}} flag. However, I have seen so many > reports > of problems with zip files, and zero reports of problems with unzipped > files, > and zero requests for zipped files, so I'm personally comfortable setting > {{{zip_ok=False}}} just to help the user avoid the problems. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pje at telecommunity.com Tue Dec 23 16:58:41 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 23 Dec 2008 10:58:41 -0500 Subject: [Distutils] Bug? zimpimporter putting in a misleading co_filename? In-Reply-To: <6cd6de210812230711p6fa7c774q3e59f09a3e954ede@mail.gmail.co m> References: <6cd6de210812221951w3d2581actf49d17305608ea13@mail.gmail.com> <6cd6de210812230711p6fa7c774q3e59f09a3e954ede@mail.gmail.com> Message-ID: <20081223155658.679A13A409D@sparrow.telecommunity.com> At 10:11 AM 12/23/2008 -0500, Rocky Bernstein wrote: >This is useful information and thanks for suggesting the workaround. > >However this does not address the issue which is not about how I can >create an unzipped egg for my use or the use of others. (Or what I >did or didn't do to create such an zipped egg). > >I am interested in writing tools that work with both zipped an >unzipped eggs. To the extent that setuptools/distutils is >responsible for setting the co_filename field in the code object >what should go inside that field? And how can a programmer reliably >and, if possible, uniformly untangle this to get/report a location. Distutils and setuptools aren't responsible. Python sets the co_filename to the original location of the source from which a file was compiled. So one way to get the source file of a function is to look in its func_globals for a __file__, and then convert .pyc/.pyo on the extension to .py. In Python 2.5 and above, the inspect and linecache modules (among others) have been modified to support zipped files; they automatically handle retrieving the source lines via the module __loader__.get_source() method as well. You may want to look at that code to see how it's handled, if you're needing to support 2.4 or 2.3 as well as 2.5+. From p.f.moore at gmail.com Tue Dec 23 17:09:21 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 23 Dec 2008 16:09:21 +0000 Subject: [Distutils] Bug? zimpimporter putting in a misleading co_filename? In-Reply-To: <20081223155658.679A13A409D@sparrow.telecommunity.com> References: <6cd6de210812221951w3d2581actf49d17305608ea13@mail.gmail.com> <6cd6de210812230711p6fa7c774q3e59f09a3e954ede@mail.gmail.com> <20081223155658.679A13A409D@sparrow.telecommunity.com> Message-ID: <79990c6b0812230809y47ac3a68ve57a4c91831973c9@mail.gmail.com> 2008/12/23 Phillip J. Eby : > At 10:11 AM 12/23/2008 -0500, Rocky Bernstein wrote: >> >> This is useful information and thanks for suggesting the workaround. >> >> However this does not address the issue which is not about how I can >> create an unzipped egg for my use or the use of others. (Or what I did or >> didn't do to create such an zipped egg). >> >> I am interested in writing tools that work with both zipped an unzipped >> eggs. To the extent that setuptools/distutils is responsible for setting the >> co_filename field in the code object what should go inside that field? And >> how can a programmer reliably and, if possible, uniformly untangle this to >> get/report a location. > > Distutils and setuptools aren't responsible. Python sets the co_filename to > the original location of the source from which a file was compiled. So one > way to get the source file of a function is to look in its func_globals for > a __file__, and then convert .pyc/.pyo on the extension to .py. > > In Python 2.5 and above, the inspect and linecache modules (among others) > have been modified to support zipped files; they automatically handle > retrieving the source lines via the module __loader__.get_source() method as > well. You may want to look at that code to see how it's handled, if you're > needing to support 2.4 or 2.3 as well as 2.5+. I just spotted this thread. I assume this is related to your question on python-dev where I asked you to clarify what you were doing :-) Phillip is correct. The inspect and linecache modules are good examples of what you need to do. But ultimately, co_filename *could* be completely arbitrary, and your code needs to deal gracefully with that worst-case situation (while handling known cases as well as it can - Nick Coghlan's mail on python-dev gives you the basic steps for that). Paul. From techtonik at gmail.com Sat Dec 27 10:11:22 2008 From: techtonik at gmail.com (anatoly techtonik) Date: Sat, 27 Dec 2008 11:11:22 +0200 Subject: [Distutils] Anybody to commit trivial fix? Message-ID: Happy Holidays! Does anybody have time to commit a trivial fix from http://bugs.python.org/issue1885 ? The issue was closed as duplicate by mistake and I do not have privileges to reopen it a new, so I am afraid it will be lost again. Thanks. -- --anatoly t. From ziade.tarek at gmail.com Sat Dec 27 10:34:59 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sat, 27 Dec 2008 10:34:59 +0100 Subject: [Distutils] Anybody to commit trivial fix? In-Reply-To: References: Message-ID: <94bdd2610812270134w7d6d1bf0i6a43e2b59e4e7736@mail.gmail.com> On Sat, Dec 27, 2008 at 10:11 AM, anatoly techtonik wrote: > Happy Holidays! Happy Holidays, > > Does anybody have time to commit a trivial fix from > http://bugs.python.org/issue1885 ? > The issue was closed as duplicate by mistake and I do not have > privileges to reopen it a new, so I am afraid it will be lost again. > It has been linked to the other so it won't be lost don't worry, I will review the various patches in the two tickets this week and work on them, Regards Tarek > Thanks. > -- > --anatoly t. > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ From techtonik at gmail.com Sat Dec 27 12:14:59 2008 From: techtonik at gmail.com (anatoly techtonik) Date: Sat, 27 Dec 2008 13:14:59 +0200 Subject: [Distutils] Anybody to commit trivial fix? In-Reply-To: <94bdd2610812270134w7d6d1bf0i6a43e2b59e4e7736@mail.gmail.com> References: <94bdd2610812270134w7d6d1bf0i6a43e2b59e4e7736@mail.gmail.com> Message-ID: On Sat, Dec 27, 2008 at 11:34 AM, Tarek Ziad? wrote: >> >> Does anybody have time to commit a trivial fix from >> http://bugs.python.org/issue1885 ? >> The issue was closed as duplicate by mistake and I do not have >> privileges to reopen it a new, so I am afraid it will be lost again. > > It has been linked to the other so it won't be lost don't worry, > I will review the various patches in the two tickets this week and work on them, > Great! The link should be broken. These issues are close, but unrelated. The 1885 is about "tar" file being overwritten and removed when several sdist archives are created at once. The 1886 is about using tarfile module to create these archives. The newest patch in superseder issue 1886 doesn't cover case for 1885 anymore. -- --anatoly t. From ziade.tarek at gmail.com Sat Dec 27 12:56:46 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sat, 27 Dec 2008 12:56:46 +0100 Subject: [Distutils] Anybody to commit trivial fix? In-Reply-To: References: <94bdd2610812270134w7d6d1bf0i6a43e2b59e4e7736@mail.gmail.com> Message-ID: <94bdd2610812270356k3a3728cbh30da26d01825ded5@mail.gmail.com> Alright, it make sense to fix it at least for the 2.6 series, I have added a unit test with your change in the ticket, and it seems to fix the bug on my platform (Mac OS X) Can you check on your platform that this test brakes when the fix is not done ? and passes when the fix is done ? Thanks Tarek On Sat, Dec 27, 2008 at 12:14 PM, anatoly techtonik wrote: > On Sat, Dec 27, 2008 at 11:34 AM, Tarek Ziad? wrote: >>> >>> Does anybody have time to commit a trivial fix from >>> http://bugs.python.org/issue1885 ? >>> The issue was closed as duplicate by mistake and I do not have >>> privileges to reopen it a new, so I am afraid it will be lost again. >> >> It has been linked to the other so it won't be lost don't worry, >> I will review the various patches in the two tickets this week and work on them, >> > > Great! The link should be broken. These issues are close, but > unrelated. The 1885 is about "tar" file being overwritten and removed > when several sdist archives are created at once. The 1886 is about > using tarfile module to create these archives. > > The newest patch in superseder issue 1886 doesn't cover case for 1885 anymore. > > -- > --anatoly t. > -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/