From netibon at justemail.net Tue Nov 1 18:23:21 2011 From: netibon at justemail.net (Ibon Net) Date: Tue, 01 Nov 2011 18:23:21 +0100 Subject: [Distutils] setuptools-0.6c11.win32-py2.7.exe installation issue Message-ID: <1320168201.4186.140660993279777@webmail.messagingengine.com> Hi, trying to get lxml for my Python-on-Windows7, I ended up installing the setuptools. But when running it, the installer complains (after the first "Next" button) "Python version 2.7 required, which was not found in the registry". Pressing "Ok" presents a table headed by "Python 2.7 is required for this package. Select installation to use:" but the table is empty, and the two fields below for python and installation directories are read-only. Of course, I do have python installed - running "Python" will give my a console window running the interpreter which states "Python 2.7.2 (default, Jun 12 2011, 14:24:26) [MSC v.1500 64 bit (AMD64)] on win32" There are entries for .py files etc in the registry - but obviously not the ones the tool is looking for... What can I do? What more do you need to know? Regards, nobi -- http://www.fastmail.fm - A fast, anti-spam email service. From ralf at systemexit.de Tue Nov 1 21:02:15 2011 From: ralf at systemexit.de (Ralf Schmitt) Date: Tue, 01 Nov 2011 21:02:15 +0100 Subject: [Distutils] setuptools-0.6c11.win32-py2.7.exe installation issue In-Reply-To: <1320168201.4186.140660993279777@webmail.messagingengine.com> (Ibon Net's message of "Tue, 01 Nov 2011 18:23:21 +0100") References: <1320168201.4186.140660993279777@webmail.messagingengine.com> Message-ID: <87vcr3a9x4.fsf@myhost.localnet> "Ibon Net" writes: > Hi, > > trying to get lxml for my Python-on-Windows7, I ended up installing the > setuptools. But when running it, the installer complains (after the > first "Next" button) > > "Python version 2.7 required, which was not found in the registry". > > Pressing "Ok" presents a table headed by > > "Python 2.7 is required for this package. Select installation to use:" > > but the table is empty, and the two fields below for python and > installation directories are read-only. > > Of course, I do have python installed - running "Python" will give my a > console window running the interpreter which states > > "Python 2.7.2 (default, Jun 12 2011, 14:24:26) [MSC v.1500 64 bit > (AMD64)] on win32" looks like you're trying to install a 32 bit installer for a 64 bit python. > > There are entries for .py files etc in the registry - but obviously not > the ones the tool is looking for... > > What can I do? What more do you need to know? > Download the source distribution and install that. -- Cheers Ralf From andrea.crotti.0 at gmail.com Wed Nov 2 10:33:42 2011 From: andrea.crotti.0 at gmail.com (Andrea Crotti) Date: Wed, 02 Nov 2011 09:33:42 +0000 Subject: [Distutils] make-like develop Message-ID: <4EB10E76.1010607@gmail.com> Supposing that we have a directory with many eggs that compose our framework. All these eggs might (or might not be) used by applications that we write. So at the moment we run "python setup.py develop" on all of them. To avoid to waste too much time, however, the builder does a make-like operation, checking if "setup.py" was modified. So since I'm rewriting a new system to build/develop: - does it make sense to re-run develop when setup.py changes. And if yes are there other possible conditions which would force me to re-run it? - what could be a good way to implement a similar mechanism? Is there anything already I could use? I've seen watchdog (http://pypi.python.org/pypi/watchdog) which looks very interesting, and might be good to implement a service, but maybe it's overkill... From jorge.vargas at gmail.com Wed Nov 2 15:02:09 2011 From: jorge.vargas at gmail.com (Jorge Vargas) Date: Wed, 2 Nov 2011 10:02:09 -0400 Subject: [Distutils] make-like develop In-Reply-To: <4EB10E76.1010607@gmail.com> References: <4EB10E76.1010607@gmail.com> Message-ID: Hi, What you are describing seems a lot like the task/way of operation of Buildout. http://pypi.python.org/pypi/zc.buildout Perhaps you should take a look at it as it's mode of operation is to maintain a collection of "eggs" either from sources or repos. Personally I like the virtualenv approach (using pip and a requirements.txt file) more but in the few projects i have used buildout it's fine. As for your other question. Yes if setup.py has changed then you must make python/distutils be aware of it. That's the only occasion where you must rerun it, at least that I know of. On Wed, Nov 2, 2011 at 5:33 AM, Andrea Crotti wrote: > Supposing that we have a directory with many eggs that compose our > framework. > > All these eggs ?might (or might not be) used by applications that we write. > So at the moment we run "python setup.py develop" on all of them. > > To avoid to waste too much time, however, the builder does a make-like > operation, checking if "setup.py" was modified. > > So since I'm rewriting a new system to build/develop: > - does it make sense to re-run develop when setup.py changes. > ?And if yes are there other possible conditions which would force me to > re-run it? > > - what could be a good way to implement a similar mechanism? > ?Is there anything already I could use? > > I've seen watchdog (http://pypi.python.org/pypi/watchdog) which looks very > interesting, > and might be good to implement a service, but maybe it's overkill... > _______________________________________________ > Distutils-SIG maillist ?- ?Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > From jim at zope.com Wed Nov 2 15:32:13 2011 From: jim at zope.com (Jim Fulton) Date: Wed, 2 Nov 2011 10:32:13 -0400 Subject: [Distutils] make-like develop In-Reply-To: <4EB10E76.1010607@gmail.com> References: <4EB10E76.1010607@gmail.com> Message-ID: On Wed, Nov 2, 2011 at 5:33 AM, Andrea Crotti wrote: ... > So since I'm rewriting a new system to build/develop: > - does it make sense to re-run develop when setup.py changes. > ?And if yes are there other possible conditions which would force me to > re-run it? You would also want to rerun develop on projects with C source code that's changed. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton From merwok at netwok.org Wed Nov 2 15:18:04 2011 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Wed, 02 Nov 2011 15:18:04 +0100 Subject: [Distutils] Odd limitation in packaging/distutils2 In-Reply-To: References: Message-ID: <4EB1511C.600@netwok.org> Hi Paul, > PS D:\Data\PyPI> D:\Data\cpython\PCbuild\python -m packaging.run install nose > installing third-party projects from an uninstalled Python is not supported There is a comment in the source to explain that: # Python would try to install into the site-packages directory under # $PREFIX, but when running from an uninstalled code checkout we don't # want to create directories under the installation root (Original bug report: #12246) > And yet after downloading the nose source... > [snip] Ah, I need to reopen the bug. The patch edited d2.install but forgot to touch the install_commands. Regards From p.f.moore at gmail.com Wed Nov 2 16:24:32 2011 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 2 Nov 2011 15:24:32 +0000 Subject: [Distutils] Odd limitation in packaging/distutils2 In-Reply-To: <4EB1511C.600@netwok.org> References: <4EB1511C.600@netwok.org> Message-ID: On 2 November 2011 14:18, ?ric Araujo wrote: > Hi Paul, > >> PS D:\Data\PyPI> D:\Data\cpython\PCbuild\python -m packaging.run install nose >> installing third-party projects from an uninstalled Python is not supported > There is a comment in the source to explain that: > > ?# Python would try to install into the site-packages directory under > ?# $PREFIX, but when running from an uninstalled code checkout we don't > ?# want to create directories under the installation root > > (Original bug report: #12246) > >> And yet after downloading the nose source... >> [snip] > Ah, I need to reopen the bug. ?The patch edited d2.install but forgot to > touch the install_commands. I'm still a bit confused. This is on Windows, and maybe things are different there, but installing seems to just work for me (except for the case where things are to be downloaded). What problem should I look for? The reason I care is that installing packages into a dev build is really useful for testing packaging changes. I'm not actually sure how to turn a dev build into an installed build on Windows... Paul. From merwok at netwok.org Wed Nov 2 16:40:36 2011 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Wed, 02 Nov 2011 16:40:36 +0100 Subject: [Distutils] Odd limitation in packaging/distutils2 In-Reply-To: References: <4EB1511C.600@netwok.org> Message-ID: <4EB16474.2020301@netwok.org> Hi, > On 2 November 2011 14:18, ?ric Araujo wrote: > I'm still a bit confused. This is on Windows, and maybe things are > different there, but installing seems to just work for me (except for > the case where things are to be downloaded). What problem should I > look for? On UNIX, the sysconfig module exposes information from the configure script, such as the install prefix. In an uninstalled built checkout, this install prefix defaults to /usr/local, so trying to install a distribution would create /usr/local/lib/python3.3/site-packages, which would be incorrect and polluting. That?s the problem I wanted to prevent. Can you define what you mean with ?just work?? Does it install things into $checkout/Lib/site-packages? > The reason I care is that installing packages into a dev build is > really useful for testing packaging changes. I'm not actually sure how > to turn a dev build into an installed build on Windows... http://docs.python.org/devguide/#quick-start : ?On Windows, load the project file PCbuild\pcbuild.sln in Visual Studio, select Debug, and Build -> Build Solution;? Cheers From andrea.crotti.0 at gmail.com Wed Nov 2 16:49:30 2011 From: andrea.crotti.0 at gmail.com (Andrea Crotti) Date: Wed, 02 Nov 2011 15:49:30 +0000 Subject: [Distutils] make-like develop In-Reply-To: References: <4EB10E76.1010607@gmail.com> Message-ID: <4EB1668A.9060106@gmail.com> On 11/02/2011 02:02 PM, Jorge Vargas wrote: > Hi, > > What you are describing seems a lot like the task/way of operation of > Buildout. http://pypi.python.org/pypi/zc.buildout Perhaps you should > take a look at it as it's mode of operation is to maintain a > collection of "eggs" either from sources or repos. > > Personally I like the virtualenv approach (using pip and a > requirements.txt file) more but in the few projects i have used > buildout it's fine. > > As for your other question. Yes if setup.py has changed then you must > make python/distutils be aware of it. That's the only occasion where > you must rerun it, at least that I know of. > Initially my idea was to actually use buildout/virtualenv. The problem is that it's harder to make them completely transparent to the user, and given the "users" I can't even possibly ask to run "bin/buildout" but there must be a script that handles everything more or less automatically... From jorge.vargas at gmail.com Wed Nov 2 16:55:10 2011 From: jorge.vargas at gmail.com (Jorge Vargas) Date: Wed, 2 Nov 2011 11:55:10 -0400 Subject: [Distutils] make-like develop In-Reply-To: <4EB1668A.9060106@gmail.com> References: <4EB10E76.1010607@gmail.com> <4EB1668A.9060106@gmail.com> Message-ID: On Wed, Nov 2, 2011 at 11:49 AM, Andrea Crotti wrote: > On 11/02/2011 02:02 PM, Jorge Vargas wrote: >> >> Hi, >> >> What you are describing seems a lot like the task/way of operation of >> Buildout. http://pypi.python.org/pypi/zc.buildout Perhaps you should >> take a look at it as it's mode of operation is to maintain a >> collection of "eggs" either from sources or repos. >> >> Personally I like the virtualenv approach (using pip and a >> requirements.txt file) more but in the few projects i have used >> buildout it's fine. >> >> As for your other question. Yes if setup.py has changed then you must >> make python/distutils be aware of it. That's the only occasion where >> you must rerun it, at least that I know of. >> > > Initially my idea was to actually use buildout/virtualenv. > The problem is that it's harder to make them completely transparent to the > user, and given the "users" I can't even possibly ask to run "bin/buildout" > but there must be a script that handles everything more or less > automatically... > As far as I know there isn't but I'll love to heard about something like this, because I do have some non-techy users that need this (web designers in my case) what we are using internally is Fabric, we can automate every call and it runs down to just running one script. But it is still a bit of trouble. From p.f.moore at gmail.com Wed Nov 2 20:27:27 2011 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 2 Nov 2011 19:27:27 +0000 Subject: [Distutils] Odd limitation in packaging/distutils2 In-Reply-To: <4EB16474.2020301@netwok.org> References: <4EB1511C.600@netwok.org> <4EB16474.2020301@netwok.org> Message-ID: On 2 November 2011 15:40, ?ric Araujo wrote: > Can you define what you mean with ?just work?? ?Does it install things > into $checkout/Lib/site-packages? Seems to. Certainly does for my own packages. >> The reason I care is that installing packages into a dev build is >> really useful for testing packaging changes. I'm not actually sure how >> to turn a dev build into an installed build on Windows... > > http://docs.python.org/devguide/#quick-start : ?On Windows, load the > project file PCbuild\pcbuild.sln in Visual Studio, select Debug, and > Build -> Build Solution;? Sorry, I wasn't clear. That I know. But that produces what I assume counts as a "non-installed" build (with python.exe in the PCBuild directory). Looking at is_python_build, it's just checking for the existence of Modules\Setup.*, so I suppose I could just rename or delete those files. It all seems a bit arbitrary, though. On a non-installed Python build (i.e., python built in a checkout) on Windows, purelib and site-packages look fine: PS D:\Data> D:\Data\cpython\PCbuild\python.exe Python 3.3.0a0 (default, Oct 28 2011, 19:56:45) [MSC v.1500 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import sysconfig, os >>> sysconfig.get_path('purelib') 'D:\\Data\\cpython\\Lib\\site-packages' >>> os.path.isdir(sysconfig.get_path('purelib')) True >>> It looks like this might be a Unix-only issue - if so, can I suggest that the restriction be removed on Windows, as it *is* useful to be able to install packages in a non-installed Python build. I'd argue that on Unix, get_path('purelib') should find the right site-packages even for an uninstalled build. But I've no idea if that's too hard, or is somehow otherwise an issue. And I don't need the functionality myself, so I'll leave that one for the Unix users to decide. Paul From merwok at netwok.org Thu Nov 3 17:45:57 2011 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Thu, 03 Nov 2011 17:45:57 +0100 Subject: [Distutils] Odd limitation in packaging/distutils2 In-Reply-To: References: <4EB1511C.600@netwok.org> <4EB16474.2020301@netwok.org> Message-ID: <4EB2C545.3080808@netwok.org> Hi, >> Can you define what you mean with ?just work?? Does it install things >> into $checkout/Lib/site-packages? > Seems to. Certainly does for my own packages. Oh, nice. The site-packages directory is not in .hgignore, so I think this works by a happy accident. >>> The reason I care is that installing packages into a dev build is >>> really useful for testing packaging changes. Agreed! With virtual environments coming to the standard library, I think the ability to install projects from an uninstalled Python will become even more useful. I will reopen the bug and try to fix sysconfig on UNIX (#12141, #6087) to remove the limitation for all OSes. >>> I'm not actually sure how to turn a dev build into an installed build >>> on Windows... >> http://docs.python.org/devguide/#quick-start : ?On Windows, load the >> project file PCbuild\pcbuild.sln in Visual Studio, select Debug, and >> Build -> Build Solution;? > > Sorry, I wasn't clear. That I know. But that produces what I assume > counts as a "non-installed" build (with python.exe in the PCBuild > directory). It?s me, I confused building and installing. *puts stupid hat on* I don?t know if Visual Studio can install a project, or if you have to turn it into a installer somehow first and then click it. You could ask on IRC or python-dev. > Looking at is_python_build, it's just checking for the existence of > Modules\Setup.*, so I suppose I could just rename or delete those > files. It all seems a bit arbitrary, though. I wouldn?t do that: It would trick Python into thinking it?s installed without actually doing all the things that make it installed. Regards From p.f.moore at gmail.com Thu Nov 3 17:53:04 2011 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 3 Nov 2011 16:53:04 +0000 Subject: [Distutils] Odd limitation in packaging/distutils2 In-Reply-To: <4EB2C545.3080808@netwok.org> References: <4EB1511C.600@netwok.org> <4EB16474.2020301@netwok.org> <4EB2C545.3080808@netwok.org> Message-ID: On 3 November 2011 16:45, ?ric Araujo wrote: > I wouldn?t do that: It would trick Python into thinking it?s installed > without actually doing all the things that make it installed. Agreed, it's not good. It's just that I don't know (on Windows) what things *do* make a Python build "installed". I'll see if I can find a definitive statement from somewhere... Paul. From merwok at netwok.org Thu Nov 3 18:13:51 2011 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Thu, 03 Nov 2011 18:13:51 +0100 Subject: [Distutils] Some distutils2/packaging questions In-Reply-To: References: Message-ID: <4EB2CBCF.3020109@netwok.org> Hi Paul, > I hope this is the right list for usage questions on > distutils2/packaging. If not, please point me at the correct list. This list is where various packaging tools are discussed and supported. In the few previous years, PEPs that are now implemented in distutils2 were discussed here before going to python-dev, for example. As you?ve seen, some distutils2 discussions that have a major impact on Python and/or where opinions from core developers are needed may also happen on python-dev. Sometimes, this list has erupted into flamewars because of setuptools fans or haters, distribute fans or haters or distutils haters. For this reason, Tarek created another list, http://groups.google.com/group/the-fellowship-of-the-packaging, that was used by the distribute developers when the project was still active (before distutils2 was started) and for distutils2 during the GSoC 2010. So if I get the picture right: - fellowship is used to discuss the implementation of distutils2 (features, bugs, code refactoring, user support) - distutils-sig should be used to discuss patches to packaging PEPs, after an agreement is reached on fellowship - python-dev can be consulted when the other MLs don?t reach agreement > I'm looking at how I'd translate existing setup.py configurations into > the new setup.cfg format. One situation I'm not sure how to deal with > is the package_data argument to setup(). This allows me to specify > data files which are to be installed as part of a package. How would I > do this in distutils2/packaging? This is currently not documented, but you can see an example in distutils2?s own setup.cfg: package_data = distutils2._backport = sysconfig.cfg distutils2.command = wininst*.exe distutils2.tests = xxmodule.c However, there is a bug that prevents this from working right now. > The resources entry seems like the intended place, but I don't see how > I can specify the destination without making too many assumptions about > precisely where the package will get installed. The resources system is intended to improve data_files and replace package_data. See the discussion on http://bugs.python.org/issue11805 . Regards From skippy.hammond at gmail.com Fri Nov 4 13:11:50 2011 From: skippy.hammond at gmail.com (Mark Hammond) Date: Fri, 04 Nov 2011 23:11:50 +1100 Subject: [Distutils] Odd limitation in packaging/distutils2 In-Reply-To: References: <4EB1511C.600@netwok.org> <4EB16474.2020301@netwok.org> <4EB2C545.3080808@netwok.org> Message-ID: <4EB3D686.4040304@gmail.com> On 4/11/2011 3:53 AM, Paul Moore wrote: > On 3 November 2011 16:45, ?ric Araujo wrote: >> I wouldn?t do that: It would trick Python into thinking it?s installed >> without actually doing all the things that make it installed. > > Agreed, it's not good. It's just that I don't know (on Windows) what > things *do* make a Python build "installed". I'll see if I can find a > definitive statement from somewhere... There really isn't anything that makes it "installed". A built Python works completely fine just where it is (and indeed, will work just fine if moved somewhere else). The closest thing to being "installed" on windows is an "InstalledPath" (or something :) registry key for the version, but that is only necessary in a limited number of contexts - when some executable other than the in-place python[w].exe needs to know where things are. I think the original report is certainly a bug on Windows (and seems mis-guided on other platforms, but each to their own :) Mark From p.f.moore at gmail.com Fri Nov 4 14:36:38 2011 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 4 Nov 2011 13:36:38 +0000 Subject: [Distutils] Odd limitation in packaging/distutils2 In-Reply-To: <4EB3D686.4040304@gmail.com> References: <4EB1511C.600@netwok.org> <4EB16474.2020301@netwok.org> <4EB2C545.3080808@netwok.org> <4EB3D686.4040304@gmail.com> Message-ID: On 4 November 2011 12:11, Mark Hammond wrote: > There really isn't anything that makes it "installed". ?A built Python works > completely fine just where it is (and indeed, will work just fine if moved > somewhere else). ?The closest thing to being "installed" on windows is an > "InstalledPath" (or something :) registry key for the version, but that is > only necessary in a limited number of contexts - when some executable other > than the in-place python[w].exe needs to know where things are. So that would imply that sysconfig.is_python_build is basically meaningless on Windows. So at a minimum, it should always return False on Windows. Actually, it's not part of the public API of sysconfig (it's not in __all__) and it's only used in one place within sysconfig.py, so maybe it should be removed and the check done inline in that one place... Paul. From merwok at netwok.org Fri Nov 4 18:25:07 2011 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Fri, 04 Nov 2011 18:25:07 +0100 Subject: [Distutils] Odd limitation in packaging/distutils2 In-Reply-To: <4EB3D686.4040304@gmail.com> References: <4EB1511C.600@netwok.org> <4EB16474.2020301@netwok.org> <4EB2C545.3080808@netwok.org> <4EB3D686.4040304@gmail.com> Message-ID: <4EB41FF3.1080304@netwok.org> Hi, Le 04/11/2011 13:11, Mark Hammond a ?crit : > On 4/11/2011 3:53 AM, Paul Moore wrote: >> Agreed, it's not good. It's just that I don't know (on Windows) what >> things *do* make a Python build "installed". I'll see if I can find a >> definitive statement from somewhere... > > There really isn't anything that makes it "installed". A built Python > works completely fine just where it is (and indeed, will work just fine > if moved somewhere else). The closest thing to being "installed" on > windows is an "InstalledPath" (or something :) registry key for the > version, but that is only necessary in a limited number of contexts - > when some executable other than the in-place python[w].exe needs to know > where things are. Ah, okay. The situation is different on UNIX: Installing Python typically writes the file in read-only mode (one consequence: users have to call setup.py --user, use virtualenv or switch to the superuser to install things), and does not just copy the build dir (so missing data files in the Makefile cause bugs that can only be seen when running the test suite or other code from an installed Python). > I think the original report is certainly a bug on Windows (and seems > mis-guided on other platforms, but each to their own :) I?ve already said that I want to remove the limitation for all OSes :) Cheers From nad at acm.org Fri Nov 4 18:38:17 2011 From: nad at acm.org (Ned Deily) Date: Fri, 04 Nov 2011 10:38:17 -0700 Subject: [Distutils] Odd limitation in packaging/distutils2 References: <4EB1511C.600@netwok.org> <4EB16474.2020301@netwok.org> <4EB2C545.3080808@netwok.org> <4EB3D686.4040304@gmail.com> <4EB41FF3.1080304@netwok.org> Message-ID: In article <4EB41FF3.1080304 at netwok.org>, ?ric Araujo wrote: > Le 04/11/2011 13:11, Mark Hammond a ?crit : > > On 4/11/2011 3:53 AM, Paul Moore wrote: > >> Agreed, it's not good. It's just that I don't know (on Windows) what > >> things *do* make a Python build "installed". I'll see if I can find a > >> definitive statement from somewhere... > > > > There really isn't anything that makes it "installed". A built Python > > works completely fine just where it is (and indeed, will work just fine > > if moved somewhere else). The closest thing to being "installed" on > > windows is an "InstalledPath" (or something :) registry key for the > > version, but that is only necessary in a limited number of contexts - > > when some executable other than the in-place python[w].exe needs to know > > where things are. > > Ah, okay. The situation is different on UNIX: Installing Python > typically writes the file in read-only mode (one consequence: users have > to call setup.py --user, use virtualenv or switch to the superuser to > install things), and does not just copy the build dir (so missing data > files in the Makefile cause bugs that can only be seen when running the > test suite or other code from an installed Python). One other very important difference: if the build was configured as --enable-shared, the final installed path to the shared library is usually captured in the built python executable. On most Unixy platforms that means you *cannot* reliably run the python executable from the build directory without taking some steps to override the default dynamic library search path, i.e. usually LD_LIBRARY_PATH. In the best case, the interpreter fails to start; in the worst, you get deceptive behavior when the newly built interpreter executable actually runs with a previously installed shared library. This is also true for OS X framework builds, which are a special kind of shared-library builds. -- Ned Deily, nad at acm.org From andrea.crotti.0 at gmail.com Sun Nov 6 13:49:22 2011 From: andrea.crotti.0 at gmail.com (Andrea Crotti) Date: Sun, 06 Nov 2011 12:49:22 +0000 Subject: [Distutils] make-like develop In-Reply-To: References: <4EB10E76.1010607@gmail.com> Message-ID: <4EB68252.8070102@gmail.com> On 11/02/2011 02:32 PM, Jim Fulton wrote: > You would also want to rerun develop on projects with C source code > that's changed. Jim Uhm true, at the moment the whole "develop" thing takes a really long time (40 seconds maybe), so we clearly need to make it much faster. If it's not completely easy to understand the conditions under which we need to re-run "develop" maybe I can do a Windows service running in background.. From h.goebel at goebel-consult.de Mon Nov 7 11:32:21 2011 From: h.goebel at goebel-consult.de (Hartmut Goebel) Date: Mon, 07 Nov 2011 11:32:21 +0100 Subject: [Distutils] make-like develop In-Reply-To: <4EB10E76.1010607@gmail.com> References: <4EB10E76.1010607@gmail.com> Message-ID: <4EB7B3B5.7050000@goebel-consult.de> Am 02.11.2011 10:33, schrieb Andrea Crotti: > So since I'm rewriting a new system to build/develop: > [...] > I've seen watchdog (http://pypi.python.org/pypi/watchdog) which looks > very interesting, > and might be good to implement a service, but maybe it's overkill... You may want to have a look at scons, a build system written in Python. It should be easier to reuse these "component" instead of developing yet-another-dependency-checking tool -- Sch?nen Gru? - Regards Hartmut Goebel Dipl.-Informatiker (univ.), CISSP, CSSLP Goebel Consult Spezialist f?r IT-Sicherheit in komplexen Umgebungen http://www.goebel-consult.de Monatliche Kolumne: http://www.cissp-gefluester.de/ Goebel Consult ist Mitglied bei http://www.7-it.de From sklein at cpcug.org Tue Nov 8 21:40:47 2011 From: sklein at cpcug.org (Stanley A. Klein) Date: Tue, 8 Nov 2011 15:40:47 -0500 (EST) Subject: [Distutils] Compatibility of bdist_rpm with Fedora packaging instructions Message-ID: <50090.71.163.198.133.1320784847.squirrel@www.cpcug.org> I will need to build some Python packages for Fedora and Centos. The spec file produced by bdist_rpm automatically includes the statement %files -f INSTALLED_FILES The Fedora Python packaging instruction includes a recommendation to avoid use of INSTALLED_FILES and provides some alternatives. That is the first incompatibility I've encountered, but there may be more. The bdist_rpm code probably should be changed to enable compatibility. Meanwhile, is there a workaround? Stan Klein From pnasrat at gmail.com Thu Nov 10 10:04:55 2011 From: pnasrat at gmail.com (Paul Nasrat) Date: Thu, 10 Nov 2011 10:04:55 +0100 Subject: [Distutils] Compatibility of bdist_rpm with Fedora packaging instructions In-Reply-To: <50090.71.163.198.133.1320784847.squirrel@www.cpcug.org> References: <50090.71.163.198.133.1320784847.squirrel@www.cpcug.org> Message-ID: I don't think bdist_rpm should track vendor packaging requirements, purely as those recommendations may change faster than the release process of distutils. I also believe bdist_rpm may be going away in the future: For Fedora have you considered rpmdev-newspec which can creates a templated python spec file for your packages. http://fedoraproject.org/wiki/How_to_create_an_RPM_package Paul On 8 November 2011 21:40, Stanley A. Klein wrote: > I will need to build some Python packages for Fedora and Centos. ?The spec > file produced by bdist_rpm automatically includes the statement > %files -f INSTALLED_FILES > > The Fedora Python packaging instruction includes a recommendation to avoid > use of INSTALLED_FILES and provides some alternatives. ?That is the first > incompatibility I've encountered, but there may be more. > > The bdist_rpm code probably should be changed to enable compatibility. > Meanwhile, is there a workaround? > > > Stan Klein > > > > > > _______________________________________________ > Distutils-SIG maillist ?- ?Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > From ziade.tarek at gmail.com Thu Nov 10 11:06:32 2011 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Thu, 10 Nov 2011 11:06:32 +0100 Subject: [Distutils] Compatibility of bdist_rpm with Fedora packaging instructions In-Reply-To: References: <50090.71.163.198.133.1320784847.squirrel@www.cpcug.org> Message-ID: On Thu, Nov 10, 2011 at 10:04 AM, Paul Nasrat wrote: > I don't think bdist_rpm should track vendor packaging requirements, > purely as those recommendations may change faster than the release > process of distutils. I also believe bdist_rpm may be going away in > the future: Yes I confirm this. We removed it in packaging because we believe it should be maintained by the RPM communities -- with their own release cycles etc. FWIW I have a custom version in the pypi2rpm project where I just feed a .spec file to the bdist_rpm command, so I can do proper RHEL or Fedora packaging. > For Fedora have you considered rpmdev-newspec which can creates a > templated python spec file for your packages. > > http://fedoraproject.org/wiki/How_to_create_an_RPM_package > > Paul > > On 8 November 2011 21:40, Stanley A. Klein wrote: >> I will need to build some Python packages for Fedora and Centos. ?The spec >> file produced by bdist_rpm automatically includes the statement >> %files -f INSTALLED_FILES >> >> The Fedora Python packaging instruction includes a recommendation to avoid >> use of INSTALLED_FILES and provides some alternatives. ?That is the first >> incompatibility I've encountered, but there may be more. >> >> The bdist_rpm code probably should be changed to enable compatibility. >> Meanwhile, is there a workaround? >> >> >> Stan Klein >> >> >> >> >> >> _______________________________________________ >> Distutils-SIG maillist ?- ?Distutils-SIG at python.org >> http://mail.python.org/mailman/listinfo/distutils-sig >> > _______________________________________________ > Distutils-SIG maillist ?- ?Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > -- Tarek Ziad? | http://ziade.org From setuptools at bugs.python.org Thu Nov 10 16:35:22 2011 From: setuptools at bugs.python.org (Sorin Sbarnea) Date: Thu, 10 Nov 2011 15:35:22 +0000 Subject: [Distutils] [issue136] Exception AttributeError: 'NoneType' raised when package is not found on Python 2.5 only Message-ID: <1320939322.2.0.248072066454.issue136@psf.upfronthosting.co.za> New submission from Sorin Sbarnea : Windows, Python 2.5 easy_install image Output: Searching for image Reading http://pypi.python.org/simple/image/ Reading https://github.com/francescortiz/image Reading https://github.com/francescortiz/image/tarball/master No local packages or download links found for image Best match: None Traceback (most recent call last): File "C:\Program Files\JetBrains\PyCharm 110.303\helpers\pydev\pydevd.py", line 1281, in debugger.run(setup['file'], None, None) File "C:\Program Files\JetBrains\PyCharm 110.303\helpers\pydev\pydevd.py", line 1010, in run pydev_imports.execfile(file, globals, locals) #execute the script File "C:/Python25/Scripts/easy_install-2.5-script.py", line 8, in load_entry_point('setuptools==0.6c11', 'console_scripts', 'easy_install-2.5')() File "C:\Python25\lib\site-packages\setuptools\command\easy_install.py", line 1712, in main with_ei_usage(lambda: File "C:\Python25\lib\site-packages\setuptools\command\easy_install.py", line 1700, in with_ei_usage return f() File "C:\Python25\lib\site-packages\setuptools\command\easy_install.py", line 1716, in distclass=DistributionWithoutHelpCommands, **kw File "C:\Python25\lib\distutils\core.py", line 151, in setup dist.run_commands() File "C:\Python25\lib\distutils\dist.py", line 974, in run_commands self.run_command(cmd) File "C:\Python25\lib\distutils\dist.py", line 994, in run_command cmd_obj.run() File "C:\Python25\lib\site-packages\setuptools\command\easy_install.py", line 211, in run self.easy_install(spec, not self.no_deps) File "C:\Python25\lib\site-packages\setuptools\command\easy_install.py", line 434, in easy_install self.local_index File "C:\Python25\lib\site-packages\setuptools\package_index.py", line 475, in fetch_distribution return dist.clone(location=self.download(dist.location, tmpdir)) AttributeError: 'NoneType' object has no attribute 'clone' ---------- messages: 648 nosy: sorin priority: bug status: unread title: Exception AttributeError: 'NoneType' raised when package is not found on Python 2.5 only _______________________________________________ Setuptools tracker _______________________________________________ From sklein at cpcug.org Thu Nov 10 18:11:51 2011 From: sklein at cpcug.org (Stanley A. Klein) Date: Thu, 10 Nov 2011 12:11:51 -0500 (EST) Subject: [Distutils] Compatibility of bdist_rpm with Fedora packaging instructions In-Reply-To: References: <50090.71.163.198.133.1320784847.squirrel@www.cpcug.org> Message-ID: <43871.71.163.198.133.1320945111.squirrel@www.cpcug.org> Tarek - I downloaded pypi2rpm and built it using bdist_rpm. The instructions on how to use it are rather thin. My best guess is to run it in spec-only mode, modify the spec, and then run it with the modified spec specified. I tried to get a help listing but that was also think and I couldn't quite figure out how to run it. Could you provide further information on how to use it. Thanks. Stan Klein On Thu, November 10, 2011 5:06 am, Tarek Ziad? wrote: > On Thu, Nov 10, 2011 at 10:04 AM, Paul Nasrat wrote: >> I don't think bdist_rpm should track vendor packaging requirements, >> purely as those recommendations may change faster than the release >> process of distutils. I also believe bdist_rpm may be going away in >> the future: > > Yes I confirm this. We removed it in packaging because we believe it > should be maintained by the RPM communities -- with their own release > cycles etc. > > FWIW I have a custom version in the pypi2rpm project where I just feed > a .spec file to the bdist_rpm command, so I can do proper RHEL or > Fedora packaging. > >> For Fedora have you considered rpmdev-newspec which can creates a >> templated python spec file for your packages. >> >> http://fedoraproject.org/wiki/How_to_create_an_RPM_package >> >> Paul >> >> On 8 November 2011 21:40, Stanley A. Klein wrote: >>> I will need to build some Python packages for Fedora and Centos. ?The >>> spec >>> file produced by bdist_rpm automatically includes the statement >>> %files -f INSTALLED_FILES >>> >>> The Fedora Python packaging instruction includes a recommendation to >>> avoid >>> use of INSTALLED_FILES and provides some alternatives. ?That is the >>> first >>> incompatibility I've encountered, but there may be more. >>> >>> The bdist_rpm code probably should be changed to enable compatibility. >>> Meanwhile, is there a workaround? >>> >>> >>> Stan Klein >>> >>> >>> >>> >>> >>> _______________________________________________ >>> Distutils-SIG maillist ?- ?Distutils-SIG at python.org >>> http://mail.python.org/mailman/listinfo/distutils-sig >>> >> _______________________________________________ >> Distutils-SIG maillist ?- ?Distutils-SIG at python.org >> http://mail.python.org/mailman/listinfo/distutils-sig >> > > > > -- > Tarek Ziad? | http://ziade.org > -- From ziade.tarek at gmail.com Thu Nov 10 18:42:08 2011 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Thu, 10 Nov 2011 18:42:08 +0100 Subject: [Distutils] Compatibility of bdist_rpm with Fedora packaging instructions In-Reply-To: <43871.71.163.198.133.1320945111.squirrel@www.cpcug.org> References: <50090.71.163.198.133.1320784847.squirrel@www.cpcug.org> <43871.71.163.198.133.1320945111.squirrel@www.cpcug.org> Message-ID: On Thu, Nov 10, 2011 at 6:11 PM, Stanley A. Klein wrote: > Tarek - > > I downloaded pypi2rpm and built it using bdist_rpm. you built it ? pypi2rpm is a script and a bidist_rpm2 command, no need to build, just install > ?The instructions on how to use it are rather thin. Yes there's no doc, as its mostly use in our own build tools for now. > My best guess is to run it in spec-only > mode, modify the spec, and then run it with the modified spec specified. > I tried to get a help listing but that was also think and I couldn't quite > figure out how to run it. > > Could you provide further information on how to use it. > > Thanks. Once it's installed you can build a rpm in your project, using: $ python setup.py --command-packages=pypi2rpm.command bdist_rpm2 --spec-file=SPECFILE (there are other options you can find with --help) > > Stan Klein > > > > On Thu, November 10, 2011 5:06 am, Tarek Ziad? wrote: >> On Thu, Nov 10, 2011 at 10:04 AM, Paul Nasrat wrote: >>> I don't think bdist_rpm should track vendor packaging requirements, >>> purely as those recommendations may change faster than the release >>> process of distutils. I also believe bdist_rpm may be going away in >>> the future: >> >> Yes I confirm this. We removed it in packaging because we believe it >> should be maintained by the RPM communities -- with their own release >> cycles etc. >> >> FWIW I have a custom version in the pypi2rpm project where I just feed >> a .spec file to the bdist_rpm command, so I can do proper RHEL or >> Fedora packaging. >> >>> For Fedora have you considered rpmdev-newspec which can creates a >>> templated python spec file for your packages. >>> >>> http://fedoraproject.org/wiki/How_to_create_an_RPM_package >>> >>> Paul >>> >>> On 8 November 2011 21:40, Stanley A. Klein wrote: >>>> I will need to build some Python packages for Fedora and Centos. ?The >>>> spec >>>> file produced by bdist_rpm automatically includes the statement >>>> %files -f INSTALLED_FILES >>>> >>>> The Fedora Python packaging instruction includes a recommendation to >>>> avoid >>>> use of INSTALLED_FILES and provides some alternatives. ?That is the >>>> first >>>> incompatibility I've encountered, but there may be more. >>>> >>>> The bdist_rpm code probably should be changed to enable compatibility. >>>> Meanwhile, is there a workaround? >>>> >>>> >>>> Stan Klein >>>> >>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> Distutils-SIG maillist ?- ?Distutils-SIG at python.org >>>> http://mail.python.org/mailman/listinfo/distutils-sig >>>> >>> _______________________________________________ >>> Distutils-SIG maillist ?- ?Distutils-SIG at python.org >>> http://mail.python.org/mailman/listinfo/distutils-sig >>> >> >> >> >> -- >> Tarek Ziad? | http://ziade.org >> > > > -- > > > -- Tarek Ziad? | http://ziade.org From sklein at cpcug.org Sat Nov 12 13:56:52 2011 From: sklein at cpcug.org (Stanley A. Klein) Date: Sat, 12 Nov 2011 07:56:52 -0500 (EST) Subject: [Distutils] More Re: Compatibility of bdist_rpm with Fedora packaging instructions Message-ID: <35590.71.163.198.133.1321102612.squirrel@www.cpcug.org> Tarek - I realized that I needed to install distutils2 to run pypi2rpm, downloaded and installed a distutils2 rpm, and tried to build rpm packages of the systems I'm working on. I downloaded the pypi sdists of the packages and tried to run rpms of them. The source includes image files as well as Python code. I'm running into a problem where the image directories are failing to be packaged. They are included in the pypi sdists, but don't get into the rpms. Any ideas on how to get the image files into the rpm packages using distutils2 and pypi2rpm? Thanks. Stan Klein On Thu, November 10, 2011 5:06 am, Tarek Ziad? wrote: > On Thu, Nov 10, 2011 at 10:04 AM, Paul Nasrat wrote: >> I don't think bdist_rpm should track vendor packaging requirements, purely as those recommendations may change faster than the release process of distutils. I also believe bdist_rpm may be going away in the future: > > Yes I confirm this. We removed it in packaging because we believe it should be maintained by the RPM communities -- with their own release cycles etc. > > FWIW I have a custom version in the pypi2rpm project where I just feed a .spec file to the bdist_rpm command, so I can do proper RHEL or Fedora packaging. > >> For Fedora have you considered rpmdev-newspec which can creates a templated python spec file for your packages. >> >> http://fedoraproject.org/wiki/How_to_create_an_RPM_package >> >> Paul >> >> On 8 November 2011 21:40, Stanley A. Klein wrote: >>> I will need to build some Python packages for Fedora and Centos. ?The spec >>> file produced by bdist_rpm automatically includes the statement %files -f INSTALLED_FILES >>> >>> The Fedora Python packaging instruction includes a recommendation to avoid >>> use of INSTALLED_FILES and provides some alternatives. ?That is the first >>> incompatibility I've encountered, but there may be more. >>> >>> The bdist_rpm code probably should be changed to enable compatibility. Meanwhile, is there a workaround? >>> >>> >>> Stan Klein >>> >>> >>> >>> >>> >>> _______________________________________________ >>> Distutils-SIG maillist ?- ?Distutils-SIG at python.org >>> http://mail.python.org/mailman/listinfo/distutils-sig >>> >> _______________________________________________ >> Distutils-SIG maillist ?- ?Distutils-SIG at python.org >> http://mail.python.org/mailman/listinfo/distutils-sig >> > > > > -- > Tarek Ziad? | http://ziade.org > -- From aclark at aclark.net Tue Nov 15 22:43:29 2011 From: aclark at aclark.net (Alex Clark) Date: Tue, 15 Nov 2011 16:43:29 -0500 Subject: [Distutils] Additional trove classifiers for Plone In-Reply-To: References: <4E9A81ED.60603@wiggy.net> <4E9A9DF6.6030607@v.loewis.de> <4E9B1219.6010606@v.loewis.de> Message-ID: Hi, On 10/17/11 9:00 AM, Alex Clark wrote: > On 10/16/11 1:19 PM, "Martin v. L?wis" wrote: >>> But let me run this by plone-dev, to get some more >>> feedback/confirmation; I'll get back to you with the exact/complete >>> list. >> >> Please also verify that they are all needed. Minimality is an objective, >> so if some plone versions are out-of-date, there is no point in adding a >> classifier. > > > OK Here is the list: > > > Framework :: Plone :: 3.3 > Framework :: Plone :: 4.0 > Framework :: Plone :: 4.1 > Framework :: Plone :: 4.2 > Framework :: Plone :: 4.3 > > > (3.3 was the first Plone version to be distributed as an egg. And 4.3 is > as far in the future as we know about right now.) > > To answer the other two questions (I found on the catalog list, sorry, I > guess I should have asked there.) > > * what specific semantics would you imply for packages tagged with these > classifiers? > > These classifiers imply compatibility with a particular Plone release. > For example, we have add-ons that only work with 4.0 (and not 4.1, etc). > > * what specific packages that are already on PyPI would be tagged with > these specific classifiers? > > Anyone releasing Plone add-ons to PyPI will now be able to specify Plone > version compatibility in setup.py. I suspect that most, if not all, > actively maintained add-ons will make use of this classifier e.g. > http://pypi.python.org/pypi/Products.PloneFormGen. But there are too > many to list. Note Plone core packages may or may not make use of these > classifiers. They could, but the request is made explicitly on behalf of > add-on developers. > > > Thank you for considering, > Any word on this? Thanks, Alex > > > Alex > > > > > > > >> >> Regards, >> Martin >> _______________________________________________ >> Distutils-SIG maillist - Distutils-SIG at python.org >> http://mail.python.org/mailman/listinfo/distutils-sig >> > > From skip at pobox.com Thu Nov 17 20:16:29 2011 From: skip at pobox.com (skip at pobox.com) Date: Thu, 17 Nov 2011 13:16:29 -0600 (CST) Subject: [Distutils] Need to install an unzipped version of an existing zipped package Message-ID: <20111117191629.CBF6320282D0@montanaro.dyndns.org> At work we have a package (MYSQL-python) which was mistakenly installed with easy_install as a zip archive. I'd like to reinstall as a non-zipped directory-based install. To guarantee that I get compile and link flags right I've been trying to experiment in my own site-packages directory but continue to be thwarted by the presence of the centrally installed zip archive version. For instance, if I execute this: export PYTHONPATH=$HOME/local/lib/python2.4/site-packages easy_install --prefix=$HOME/local -Z -U MYSQL-python it executes very quickly, with no obvious attempts to compile or link anything. When I look at the easy-install.pth file in my site-packages directory I find that all it did was drop in a reference to the centrally installed zip archive version of the package! That's not what I wanted at all. How can I tell easy_install to ignore that centrally installed version? Failing that, is it okay to simply unzip the package we have provided in the end it has the same name? Thanks, -- Skip Montanaro - skip at pobox.com - http://www.smontanaro.net/ From techtonik at gmail.com Fri Nov 18 14:41:12 2011 From: techtonik at gmail.com (anatoly techtonik) Date: Fri, 18 Nov 2011 16:41:12 +0300 Subject: [Distutils] setuptools-0.6c11.win32-py2.7.exe installation issue In-Reply-To: <87vcr3a9x4.fsf@myhost.localnet> References: <1320168201.4186.140660993279777@webmail.messagingengine.com> <87vcr3a9x4.fsf@myhost.localnet> Message-ID: -- anatoly t. On Tue, Nov 1, 2011 at 11:02 PM, Ralf Schmitt wrote: > "Ibon Net" writes: > > > Hi, > > > > trying to get lxml for my Python-on-Windows7, I ended up installing the > > setuptools. But when running it, the installer complains (after the > > first "Next" button) > > > > "Python version 2.7 required, which was not found in the registry". > > > > Pressing "Ok" presents a table headed by > > > > "Python 2.7 is required for this package. Select installation to use:" > > > > but the table is empty, and the two fields below for python and > > installation directories are read-only. > > > > Of course, I do have python installed - running "Python" will give my a > > console window running the interpreter which states > > > > "Python 2.7.2 (default, Jun 12 2011, 14:24:26) [MSC v.1500 64 bit > > (AMD64)] on win32" > > looks like you're trying to install a 32 bit installer for a 64 bit > python. > > > > > There are entries for .py files etc in the registry - but obviously not > > the ones the tool is looking for... > > > > What can I do? What more do you need to know? > > > > Download the source distribution and install that. Wait. Why not to fix this in distutils itself? http://code.google.com/p/pythonxy/source/detail?r=ddeb3982593a0569bcc0fc392b56cbf65962491d&repo=xy-27 -- anatoly t. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf at systemexit.de Mon Nov 21 01:16:05 2011 From: ralf at systemexit.de (Ralf Schmitt) Date: Mon, 21 Nov 2011 01:16:05 +0100 Subject: [Distutils] [ANNOUNCE] pypiserver 0.4.0 - minimal pypi server Message-ID: <87ehx2ba9m.fsf@myhost.localnet> Hi, I've just uploaded pypiserver 0.4.0 to the python package index. pypiserver is a minimal PyPI compatible server. It can be used to serve a set of packages and eggs to easy_install or pip. pypiserver is easy to install (i.e. just easy_install pypiserver). It doesn't have any external dependencies. http://pypi.python.org/pypi/pypiserver/ should contain enough information to easily get you started running your own PyPI server in a few minutes. The code is available on github: https://github.com/schmir/pypiserver Changes in version 0.4.0 ------------------------- - add functionality to manage package updates - updated documentation - python 3 support has been added -- Cheers, Ralf From oscar.benjamin at bristol.ac.uk Wed Nov 23 12:48:36 2011 From: oscar.benjamin at bristol.ac.uk (Oscar Benjamin) Date: Wed, 23 Nov 2011 11:48:36 +0000 Subject: [Distutils] linking with libm Message-ID: <20111123114836.GA3324@ENM-OB> Hello, I have an extension module that uses math.h. In order to compile it on Linux with gcc, it needs to link with libm. However, using libraries=['m'] leads to a linker error when using mingw32 on Windows. At the moment my setup.py contains something like: libraries = [] if not 'win' in sys.platform: libraries.append('m') This works for the two cases I user personally. I'm guessing, though, that I should really be checking for something more specific, perhaps to do with the compiler that is being used. I guess that using math.h and wanting cross-platform compilation is probably quite a common case. So I wondered if anyone who knows more than me about c-compilers and distutils has a better suggestion for this. Apologies if this is mentioned somewhere already. The only thing I could find in the docs was instructions for adding linker options when installing python modules. Ideally, I'd like to be able to handle this in my setup.py without needing the setup.py user to manually alter the linker settings. Thanks, Oscar. From ralf at systemexit.de Wed Nov 23 13:22:00 2011 From: ralf at systemexit.de (Ralf Schmitt) Date: Wed, 23 Nov 2011 13:22:00 +0100 Subject: [Distutils] linking with libm In-Reply-To: <20111123114836.GA3324@ENM-OB> (Oscar Benjamin's message of "Wed, 23 Nov 2011 11:48:36 +0000") References: <20111123114836.GA3324@ENM-OB> Message-ID: <87k46r111z.fsf@winserver.brainbot.com> Oscar Benjamin writes: > Hello, > > I have an extension module that uses math.h. In order to compile it on > Linux with gcc, it needs to link with libm. However, using > libraries=['m'] leads to a linker error when using mingw32 on Windows. > > At the moment my setup.py contains something like: > > libraries = [] > if not 'win' in sys.platform: > libraries.append('m') that fails on OS X - if I remember correctly > > This works for the two cases I user personally. I'm guessing, though, > that I should really be checking for something more specific, perhaps to > do with the compiler that is being used. I guess that using math.h and > wanting cross-platform compilation is probably quite a common case. So I > wondered if anyone who knows more than me about c-compilers and > distutils has a better suggestion for this. > > Apologies if this is mentioned somewhere already. The only thing I could > find in the docs was instructions for adding linker options when > installing python modules. Ideally, I'd like to be able to handle this > in my setup.py without needing the setup.py user to manually alter the > linker settings. I'm not quite sure if it's the right way to do it, but I solved that with the following code: ,---- | from distutils import sysconfig | | if sysconfig.get_config_var("LIBM") == "-lm": | libraries = ["m"] | else: | libraries = [] `---- (e.g. in https://github.com/pediapress/timelib/commit/70a8a9c42ff005fbb3547cc6b300da05a42d0cf2) -- Cheers Ralf From oscar.benjamin at bristol.ac.uk Wed Nov 23 15:30:48 2011 From: oscar.benjamin at bristol.ac.uk (Oscar Benjamin) Date: Wed, 23 Nov 2011 14:30:48 +0000 Subject: [Distutils] linking with libm In-Reply-To: <87k46r111z.fsf@winserver.brainbot.com> References: <20111123114836.GA3324@ENM-OB> <87k46r111z.fsf@winserver.brainbot.com> Message-ID: <20111123143047.GA2584@ENM-OB> > ,---- > | from distutils import sysconfig > | > | if sysconfig.get_config_var("LIBM") == "-lm": > | libraries = ["m"] > | else: > | libraries = [] > `---- > Thanks, that does look better. If I understand correctly that snippet will check to see if python was linked with libm and ensure that my extension module will also be only if python was. I tested it with Linux and Windows and it works and seems a lot more robust than what I had before. Would it be slightly better to test just for the existence of the LIBM configuration variable rather than its value? I don't know of an example, but I guess a system could have libm but use a compiler with a different linker argument format. e.g. if 'libm' in sysconfig.get_config_vars() Is there a more general way to discover if the compiler that is going to be used knows of a library with a specified name? Something like if get_compiler_type().library_exists(libname) Thanks, Oscar. On Wed, Nov 23, 2011 at 01:22:00PM +0100, Ralf Schmitt wrote: > Oscar Benjamin writes: > > > Hello, > > > > I have an extension module that uses math.h. In order to compile it on > > Linux with gcc, it needs to link with libm. However, using > > libraries=['m'] leads to a linker error when using mingw32 on Windows. > > > > At the moment my setup.py contains something like: > > > > libraries = [] > > if not 'win' in sys.platform: > > libraries.append('m') > > that fails on OS X - if I remember correctly > > > > > This works for the two cases I user personally. I'm guessing, though, > > that I should really be checking for something more specific, perhaps to > > do with the compiler that is being used. I guess that using math.h and > > wanting cross-platform compilation is probably quite a common case. So I > > wondered if anyone who knows more than me about c-compilers and > > distutils has a better suggestion for this. > > > > Apologies if this is mentioned somewhere already. The only thing I could > > find in the docs was instructions for adding linker options when > > installing python modules. Ideally, I'd like to be able to handle this > > in my setup.py without needing the setup.py user to manually alter the > > linker settings. > > I'm not quite sure if it's the right way to do it, but I solved that > with the following code: > > ,---- > | from distutils import sysconfig > | > | if sysconfig.get_config_var("LIBM") == "-lm": > | libraries = ["m"] > | else: > | libraries = [] > `---- > > (e.g. in https://github.com/pediapress/timelib/commit/70a8a9c42ff005fbb3547cc6b300da05a42d0cf2) > > -- > Cheers > Ralf From ralf at systemexit.de Wed Nov 23 16:07:43 2011 From: ralf at systemexit.de (Ralf Schmitt) Date: Wed, 23 Nov 2011 16:07:43 +0100 Subject: [Distutils] linking with libm In-Reply-To: <20111123143047.GA2584@ENM-OB> (Oscar Benjamin's message of "Wed, 23 Nov 2011 14:30:48 +0000") References: <20111123114836.GA3324@ENM-OB> <87k46r111z.fsf@winserver.brainbot.com> <20111123143047.GA2584@ENM-OB> Message-ID: <87bos227y8.fsf@winserver.brainbot.com> Oscar Benjamin writes: > Would it be slightly better to test just for the existence of the LIBM > configuration variable rather than its value? I don't know of an > example, but I guess a system could have libm but use a compiler with a > different linker argument format. > > e.g. > if 'libm' in sysconfig.get_config_vars() No, the following is from OS X: ,---- | >>> from distutils import sysconfig | >>> sysconfig.get_config_var("LIBM") | '' `---- > > Is there a more general way to discover if the compiler that is going to > be used knows of a library with a specified name? Something like > > if get_compiler_type().library_exists(libname) It's probably possible with some custom distutils command, but it's also not worth it IMHO. From sienkiew at stsci.edu Wed Nov 23 23:42:40 2011 From: sienkiew at stsci.edu (Mark Sienkiewicz) Date: Wed, 23 Nov 2011 17:42:40 -0500 Subject: [Distutils] linking with libm In-Reply-To: <87bos227y8.fsf@winserver.brainbot.com> References: <20111123114836.GA3324@ENM-OB> <87k46r111z.fsf@winserver.brainbot.com> <20111123143047.GA2584@ENM-OB> <87bos227y8.fsf@winserver.brainbot.com> Message-ID: <4ECD76E0.3070605@stsci.edu> On 11/23/11 10:07, Ralf Schmitt wrote: > Oscar Benjamin writes: > >> Would it be slightly better to test just for the existence of the LIBM >> configuration variable rather than its value? I don't know of an >> example, but I guess a system could have libm but use a compiler with a >> different linker argument format. >> >> e.g. >> if 'libm' in sysconfig.get_config_vars() > No, the following is from OS X: > ,---- > |>>> from distutils import sysconfig > |>>> sysconfig.get_config_var("LIBM") > | '' > `---- > It may not matter. On my Mac (Snow Leopard), it wants to use built-in functions provided by gcc for everything that I've tried that is in -lm on other machines. There is a /usr/lib/libm.dylib, but a little searching (/bin, /usr/bin, my own custom python install) did not find anything on my machine that actually uses it. "cc c.c" and "cc c.c -lm" produce identical a.out files. From andrea.crotti.0 at gmail.com Thu Nov 24 11:54:33 2011 From: andrea.crotti.0 at gmail.com (Andrea Crotti) Date: Thu, 24 Nov 2011 10:54:33 +0000 Subject: [Distutils] easy_install not working Message-ID: <4ECE2269.4040208@gmail.com> Since some days easy_install doesn't work correctly anymore. I get for example something similar. Any idea of what it could be? In general setuptools is working perfectly, only easy_install is not working anymore.. [andrea at precision ~]$ easy_install-2.7 --user python-graph-core Traceback (most recent call last): File "/home/andrea/.local/bin/easy_install-2.7", line 5, in from pkg_resources import load_entry_point File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 2711, in parse_requirements(__requires__), Environment() File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 584, in resolve raise DistributionNotFound(req) pkg_resources.DistributionNotFound: distribute==0.6.21 From ziade.tarek at gmail.com Thu Nov 24 12:44:15 2011 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Thu, 24 Nov 2011 12:44:15 +0100 Subject: [Distutils] easy_install not working In-Reply-To: <4ECE2269.4040208@gmail.com> References: <4ECE2269.4040208@gmail.com> Message-ID: Please try to update your installation to the latest revision of Distribute. You currently have 0.6.21 and it seems that somehow it's in a broken state $ easy_install-2.7 -U Distribute (with sudo rights if needed) On Thu, Nov 24, 2011 at 11:54 AM, Andrea Crotti wrote: > Since some days easy_install doesn't work correctly anymore. > I get for example something similar. > Any idea of what it could be? > In general setuptools is working perfectly, only easy_install is not working > anymore.. > > [andrea at precision ~]$ easy_install-2.7 --user python-graph-core > Traceback (most recent call last): > ?File "/home/andrea/.local/bin/easy_install-2.7", line 5, in > ? ?from pkg_resources import load_entry_point > ?File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 2711, in > > ? ?parse_requirements(__requires__), Environment() > ?File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 584, in > resolve > ? ?raise DistributionNotFound(req) > pkg_resources.DistributionNotFound: distribute==0.6.21 > > _______________________________________________ > Distutils-SIG maillist ?- ?Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > -- Tarek Ziad? | http://ziade.org From hanno at hannosch.eu Thu Nov 24 12:49:07 2011 From: hanno at hannosch.eu (Hanno Schlichting) Date: Thu, 24 Nov 2011 12:49:07 +0100 Subject: [Distutils] easy_install not working In-Reply-To: References: <4ECE2269.4040208@gmail.com> Message-ID: On Thu, Nov 24, 2011 at 12:44 PM, Tarek Ziad? wrote: > Please try to update your installation to the latest revision of > Distribute. You currently have 0.6.21 and it seems that somehow it's > in a broken state > > $ easy_install-2.7 -U Distribute That should probably be a lower case d: $ easy_install-2.7 -U distribute > (with sudo rights if needed) Hanno From andrea.crotti.0 at gmail.com Fri Nov 25 11:04:24 2011 From: andrea.crotti.0 at gmail.com (Andrea Crotti) Date: Fri, 25 Nov 2011 10:04:24 +0000 Subject: [Distutils] easy_install not working In-Reply-To: References: <4ECE2269.4040208@gmail.com> Message-ID: <4ECF6828.7090702@gmail.com> On 11/24/2011 11:44 AM, Tarek Ziad? wrote: > Please try to update your installation to the latest revision of > Distribute. You currently have 0.6.21 and it seems that somehow it's > in a broken state > > $ easy_install-2.7 -U Distribute > > (with sudo rights if needed) > > Mm no it doesn't let me do it, returning the same error... I might have messed something maybe then, not sure what exactly. Can the easy_install.pth influence this? From andrea.crotti.0 at gmail.com Fri Nov 25 11:06:12 2011 From: andrea.crotti.0 at gmail.com (Andrea Crotti) Date: Fri, 25 Nov 2011 10:06:12 +0000 Subject: [Distutils] develop and --record Message-ID: <4ECF6894.3090606@gmail.com> So "setup.py develop" also provides the --record option, but apparently it doesn't record anything if I use it. At least it should record the creation of scripts in case there are some, but the record file results always empty. Is that the normal behaviour? I just want to make sure that I can easily clean up everything what I added to .local/bin for example. From ziade.tarek at gmail.com Fri Nov 25 11:25:39 2011 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Fri, 25 Nov 2011 11:25:39 +0100 Subject: [Distutils] easy_install not working In-Reply-To: <4ECF6828.7090702@gmail.com> References: <4ECE2269.4040208@gmail.com> <4ECF6828.7090702@gmail.com> Message-ID: On Fri, Nov 25, 2011 at 11:04 AM, Andrea Crotti wrote: .. > Mm no it doesn't let me do it, returning the same error... > I might have messed something maybe then, not sure what exactly. > Can the easy_install.pth influence this? I don't know, maybe it's related to the per-user environment but I am not sure. What you can do to make sure you have a clean central install (after a backup of site-packages) is: 1/ remove any occurences of those in site-packages (egg dirs, code dirs) - setuptools* - distribute* - pkg_resources* 2/ open easy_install.pth and remove any line starting with setutptools or distribute 3/ do a fresh install of Distribute (since that's the one you initially had) $ wget http://python-distribute.org/distribute_setup.py $ sudo python distribute_setup.py If you still have the issue, try to backup your user's local python dir, remove it and try again with the --user option HTH Tarek -- Tarek Ziad? | http://ziade.org From ziade.tarek at gmail.com Fri Nov 25 11:32:57 2011 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Fri, 25 Nov 2011 11:32:57 +0100 Subject: [Distutils] develop and --record In-Reply-To: <4ECF6894.3090606@gmail.com> References: <4ECF6894.3090606@gmail.com> Message-ID: On Fri, Nov 25, 2011 at 11:06 AM, Andrea Crotti wrote: > So "setup.py develop" also provides the --record option, but apparently it > doesn't > record anything if I use it. well, since develop just hooks a package in-place, I don't think the --record option does anything > > At least it should record the creation of scripts in case there are some, > but the record > file results always empty. > Is that the normal behaviour? > > I just want to make sure that I can easily clean up everything what I added > to .local/bin > for example. That's a good point. Please add a ticket here : https://bitbucket.org/tarek/distribute/issues > _______________________________________________ > Distutils-SIG maillist ?- ?Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > -- Tarek Ziad? | http://ziade.org From andrea.crotti.0 at gmail.com Fri Nov 25 11:39:55 2011 From: andrea.crotti.0 at gmail.com (Andrea Crotti) Date: Fri, 25 Nov 2011 10:39:55 +0000 Subject: [Distutils] develop and --record In-Reply-To: References: <4ECF6894.3090606@gmail.com> Message-ID: <4ECF707B.5090401@gmail.com> On 11/25/2011 10:32 AM, Tarek Ziad? wrote: > well, since develop just hooks a package in-place, I don't think the > --record option does anything It should at least record these files for example: Installing gen_requires.py script to /home/andrea/.local/bin Installing remove_eggs.py script to /home/andrea/.local/bin Installing dev_main_no_eggs.py script to /home/andrea/.local/bin And also the link created, which is still a file written in site-packages imho.. > That's a good point. Please add a ticket here : > https://bitbucket.org/tarek/distribute/issues > I will open a ticket sure, I can't rely however on the development version, so I'll need another way to fix this maybe.. From andrea.crotti.0 at gmail.com Fri Nov 25 15:12:50 2011 From: andrea.crotti.0 at gmail.com (Andrea Crotti) Date: Fri, 25 Nov 2011 14:12:50 +0000 Subject: [Distutils] develop and --record In-Reply-To: References: <4ECF6894.3090606@gmail.com> Message-ID: <4ECFA262.1090704@gmail.com> Another thing which is more or less related to this. Suppose that I'm working on module a.py, which will generate a a.pyc. Now when I move a.py to b.py the a.pyc will still be there, and might make work all the imports to "a", even if they are not supposed to. Since with egg-info I can see the current sources, maybe is there a smart way to clean these unwanted pyc files? From setuptools at bugs.python.org Fri Nov 25 19:19:33 2011 From: setuptools at bugs.python.org (Zooko O'Whielacronx) Date: Fri, 25 Nov 2011 18:19:33 +0000 Subject: [Distutils] [issue137] test_get_script_header_jython_workaround expects output on stdout, but it is on stderr instead Message-ID: <1322245173.76.0.414739422091.issue137@psf.upfronthosting.co.za> New submission from Zooko O'Whielacronx : This unit test fails on CPython 2.7.1 and pypy 1.7: FAIL: test_get_script_header_jython_workaround (setuptools.tests.test_resources.ScriptHeaderTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/zooko/playground/setuptools/setuptools/tests/test_resources.py", line 524, in test_get_script_header_jython_workaround self.assert_('Unable to adapt shebang line' in sys.stdout.getvalue()) AssertionError: False is not true The following patch makes it pass on those two pythons: Index: setuptools/tests/test_resources.py =================================================================== --- setuptools/tests/test_resources.py (revision 88925) +++ setuptools/tests/test_resources.py (working copy) @@ -507,7 +507,7 @@ def test_get_script_header_jython_workaround(self): platform = sys.platform sys.platform = 'java1.5.0_13' - stdout = sys.stdout + stderr = sys.stderr try: # A mock sys.executable that uses a shebang line (this file) exe = os.path.normpath(os.path.splitext(__file__)[0] + '.py') @@ -517,17 +517,17 @@ # Ensure we generate what is basically a broken shebang line # when there's options, with a warning emitted - sys.stdout = StringIO.StringIO() + sys.stderr = StringIO.StringIO() self.assertEqual(get_script_header('#!/usr/bin/python -x', executable=exe), '#!%s -x\n' % exe) - self.assert_('Unable to adapt shebang line' in sys.stdout.getvalue()) - sys.stdout = StringIO.StringIO() + self.assert_('Unable to adapt shebang line' in sys.stderr.getvalue()) + sys.stderr = StringIO.StringIO() self.assertEqual(get_script_header('#!/usr/bin/python', executable=self.non_ascii_exe), '#!%s -x\n' % self.non_ascii_exe) - self.assert_('Unable to adapt shebang line' in sys.stdout.getvalue()) + self.assert_('Unable to adapt shebang line' in sys.stderr.getvalue()) finally: sys.platform = platform - sys.stdout = stdout + sys.stderr = stderr ---------- messages: 653 nosy: zooko priority: bug status: unread title: test_get_script_header_jython_workaround expects output on stdout, but it is on stderr instead _______________________________________________ Setuptools tracker _______________________________________________ From cybersation at hotmail.com Fri Nov 18 15:08:48 2011 From: cybersation at hotmail.com (Digital girl) Date: Fri, 18 Nov 2011 09:08:48 -0500 Subject: [Distutils] 'Easy Install' Question In-Reply-To: References: Message-ID: Hi, I was really hoping someone can help me out. I ran the ez_setup python script from here: http://peak.telecommunity.com/dist/ez_setup.py then edited the path variable to include path to python directory. I can run python from command prompt but when I try to run easy_install it invokes a Python interpreter window. Please see screen capture. I'm relatively new to Python and still learning syntax and semantics. Can someone advise as to what I need to do next to get easy_install installed correctly. I did peruse the discussion threads but didn't see anything that addressed my issue. Thanks in advance. Regards, Amanda -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: easyInstall.png Type: image/png Size: 130462 bytes Desc: not available URL: From pje at telecommunity.com Sat Nov 26 18:35:16 2011 From: pje at telecommunity.com (PJ Eby) Date: Sat, 26 Nov 2011 12:35:16 -0500 Subject: [Distutils] 'Easy Install' Question In-Reply-To: References: Message-ID: On Fri, Nov 18, 2011 at 9:08 AM, Digital girl wrote: > Hi, > > I was really hoping someone can help me out. I ran the ez_setup python > script from here: > > http://peak.telecommunity.com/dist/ez_setup.py > > then edited the path variable to include path to python directory. > > I can run python from command prompt but when I try to run easy_install it > invokes a Python interpreter window. Please see screen capture. > The problem is that you ran the script from PythonWin, so the script thought it was your Python interpreter. That's why it comes up the way it does in your screenshot. Install it again, but instead of using PythonWin to run the script, run it from the command line like this: C:\Python27> python ez_setup.py Then it will configure easy_install use python.exe instead of pythonwin.exe, and it won't dump you into PythonWin like that any more. -------------- next part -------------- An HTML attachment was scrubbed... URL: From sklein at cpcug.org Sat Nov 26 20:00:01 2011 From: sklein at cpcug.org (Stanley A. Klein) Date: Sat, 26 Nov 2011 14:00:01 -0500 (EST) Subject: [Distutils] Compatibility of bdist_rpm with Fedora packaging instructions In-Reply-To: References: Message-ID: <52526.71.163.68.30.1322334001.squirrel@www.cpcug.org> Tarek - Thanks. I got it working and want to document some of my findings. I'm cc:'ing the Fedora Python list so they can take these issues into account in their Python packaging instructions. My findings are as follows: 1. The source code management system was git. I needed to install setuptools-git to get files recognized that were being maintained under git. 2. I also needed to establish a MANIFEST.in file to ensure all relevant files were included. 3. The setup.cfg statement under [bdist_rpm] of "doc_files =" doesn't work if there are directories involved. This is a documented "gotcha" in http://fedoraproject.org/wiki/How_to_create_an_RPM_package They also advise avoiding use of INSTALLED_FILES. Here is what I used in the spec file (it had to be edited for the directories to be included in docs): %install python setup.py install --root=$RPM_BUILD_ROOT cd $RPM_BUILD_ROOT mkdir -p %{buildroot}%{_defaultdocdir}/%{name}-%{version}/ mkdir -p %{buildroot}%{_defaultdocdir}/%{name}-%{version}/docs mkdir -p %{buildroot}%{_defaultdocdir}/%{name}-%{version}/examples cd %{_builddir}/%{name}-%{version} cp -p *.txt %{buildroot}%{_defaultdocdir}/%{name}-%{version}/ cp -rp docs/ %{buildroot}%{_defaultdocdir}/%{name}-%{version}/docs cp -rp examples/ %{buildroot}%{_defaultdocdir}/%{name}-%{version}/examples %clean rm -rf $RPM_BUILD_ROOT %files %defattr(-,root,root) %{_defaultdocdir}/%{name}-%{version}/ %{python_sitelib}/%{name}/ %{python_sitelib}/%{name}-%{version}-py%{python_version}.egg-info/ 4. The overall approach I used was to run $ python setup.py --command-packages=pypi2rpm.command bdist_rpm2 --spec-only That brought in items such as the description specified in setup.py. I then moved the spec file up a directory level from dist, edited in the above parts relevant to the doc files, and finally ran $ python setup.py --command-packages=pypi2rpm.command bdist_rpm2 > --spec-file=SPECFILE I also found it very useful to actually run python setup.py --command-packages=pypi2rpm.command bdist_rpm2 --spec-file SPECFILE >buildout 2>&1 Recording the build output enabled me to find details of errors that occurred during the build. There were a number of those, and the output flies by too quickly to notice them for diagnosis of problems that need to be fixed. (an approach I've also used in the past is to tee the output, so I can see if the build was successful when it completes). Also, the -k option came in useful in figuring out the documentation-related fixes to the spec file, because it helped seeing what was included in the build and what was available in the build but not getting copied to the install. 5. I inquired about how the project produced its sdist files for pypi, and was told that they had so many problems related to their switch to git that they simply did a gzipped tar of their repository. 6. The project used a different extension on their README file from the expected values of README and README.txt. I needed to change that to README.txt to get it to process properly. Again, thanks. Stan Klein On Fri, November 11, 2011 6:00 am, Tarek Ziad? wrote: > > Message: 3 > Date: Thu, 10 Nov 2011 18:42:08 +0100 > From: Tarek Ziad? > To: "Stanley A. Klein" > Cc: distutils-sig at python.org > Subject: Re: [Distutils] Compatibility of bdist_rpm with Fedora > packaging instructions > > On Thu, Nov 10, 2011 at 6:11 PM, Stanley A. Klein > wrote: >> Tarek - >> >> I downloaded pypi2rpm and built it using bdist_rpm. > > you built it ? pypi2rpm is a script and a bidist_rpm2 command, no need > to build, just install > > >> ?The instructions on how to use it are rather thin. > > Yes there's no doc, as its mostly use in our own build tools for now. > >> My best guess is to run it in spec-only >> mode, modify the spec, and then run it with the modified spec specified. >> I tried to get a help listing but that was also think and I couldn't >> quite >> figure out how to run it. >> >> Could you provide further information on how to use it. >> >> Thanks. > > Once it's installed you can build a rpm in your project, using: > > $ python setup.py --command-packages=pypi2rpm.command bdist_rpm2 > --spec-file=SPECFILE > > (there are other options you can find with --help) > > >> >> Stan Klein >> >> >> >> On Thu, November 10, 2011 5:06 am, Tarek Ziad? wrote: >>> On Thu, Nov 10, 2011 at 10:04 AM, Paul Nasrat >>> wrote: >>>> I don't think bdist_rpm should track vendor packaging requirements, >>>> purely as those recommendations may change faster than the release >>>> process of distutils. I also believe bdist_rpm may be going away in >>>> the future: >>> >>> Yes I confirm this. We removed it in packaging because we believe it >>> should be maintained by the RPM communities -- with their own release >>> cycles etc. >>> >>> FWIW I have a custom version in the pypi2rpm project where I just feed >>> a .spec file to the bdist_rpm command, so I can do proper RHEL or >>> Fedora packaging. >>> >>>> For Fedora have you considered rpmdev-newspec which can creates a >>>> templated python spec file for your packages. >>>> >>>> http://fedoraproject.org/wiki/How_to_create_an_RPM_package >>>> >>>> Paul >>>> >>>> On 8 November 2011 21:40, Stanley A. Klein wrote: >>>>> I will need to build some Python packages for Fedora and Centos. ?The >>>>> spec >>>>> file produced by bdist_rpm automatically includes the statement >>>>> %files -f INSTALLED_FILES >>>>> >>>>> The Fedora Python packaging instruction includes a recommendation to >>>>> avoid >>>>> use of INSTALLED_FILES and provides some alternatives. ?That is the >>>>> first >>>>> incompatibility I've encountered, but there may be more. >>>>> >>>>> The bdist_rpm code probably should be changed to enable >>>>> compatibility. >>>>> Meanwhile, is there a workaround? >>>>> >>>>> >>>>> Stan Klein From ziade.tarek at gmail.com Sat Nov 26 22:24:42 2011 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sat, 26 Nov 2011 22:24:42 +0100 Subject: [Distutils] Compatibility of bdist_rpm with Fedora packaging instructions In-Reply-To: <52526.71.163.68.30.1322334001.squirrel@www.cpcug.org> References: <52526.71.163.68.30.1322334001.squirrel@www.cpcug.org> Message-ID: Thanks for the feedback Stanley ! If you agree, I'll add your document into the pypi2rpm/bdist_rpm2 project, for other people Cheers Tarek On Sat, Nov 26, 2011 at 8:00 PM, Stanley A. Klein wrote: > Tarek - > > Thanks. > > I got it working and want to document some of my findings. ?I'm cc:'ing > the Fedora Python list so they can take these issues into account in their > Python packaging instructions. ?My findings are as follows: > > 1. ?The source code management system was git. ?I needed to install > setuptools-git to get files recognized that were being maintained under > git. > > 2. ?I also needed to establish a MANIFEST.in file to ensure all relevant > files were included. > > 3. ?The setup.cfg statement under [bdist_rpm] of "doc_files =" doesn't > work if there are directories involved. ?This is a documented "gotcha" in > http://fedoraproject.org/wiki/How_to_create_an_RPM_package > > They also advise avoiding use of INSTALLED_FILES. > > Here is what I used in the spec file (it had to be edited for the > directories to be included in docs): > > %install > python setup.py install --root=$RPM_BUILD_ROOT > cd $RPM_BUILD_ROOT > mkdir -p %{buildroot}%{_defaultdocdir}/%{name}-%{version}/ > mkdir -p %{buildroot}%{_defaultdocdir}/%{name}-%{version}/docs > mkdir -p %{buildroot}%{_defaultdocdir}/%{name}-%{version}/examples > cd ?%{_builddir}/%{name}-%{version} > cp -p *.txt %{buildroot}%{_defaultdocdir}/%{name}-%{version}/ > cp -rp docs/ %{buildroot}%{_defaultdocdir}/%{name}-%{version}/docs > cp -rp examples/ %{buildroot}%{_defaultdocdir}/%{name}-%{version}/examples > > %clean > rm -rf $RPM_BUILD_ROOT > > %files > %defattr(-,root,root) > %{_defaultdocdir}/%{name}-%{version}/ > %{python_sitelib}/%{name}/ > %{python_sitelib}/%{name}-%{version}-py%{python_version}.egg-info/ > > 4. ?The overall approach I used was to run > > $ python setup.py --command-packages=pypi2rpm.command bdist_rpm2 > ?--spec-only > > That brought in items such as the description specified in setup.py. ?I > then moved the spec file up a directory level from dist, edited in the > above parts relevant to the doc files, and finally ran > > $ python setup.py --command-packages=pypi2rpm.command bdist_rpm2 >> --spec-file=SPECFILE > > I also found it very useful to actually run > > python setup.py --command-packages=pypi2rpm.command bdist_rpm2 --spec-file > SPECFILE ? >buildout 2>&1 > > Recording the build output enabled me to find details of errors that > occurred during the build. ?There were a number of those, and the output > flies by too quickly to notice them for diagnosis of problems that need to > be fixed. ?(an approach I've also used in the past is to tee the output, > so I can see if the build was successful when it completes). ?Also, the -k > option came in useful in figuring out the documentation-related fixes to > the spec file, because it helped seeing what was included in the build and > what was available in the build but not getting copied to the install. > > 5. ?I inquired about how the project produced its sdist files for pypi, > and was told that they had so many problems related to their switch to git > that they simply did a gzipped tar of their repository. > > 6. ?The project used a different extension on their README file from the > expected values of README and README.txt. ?I needed to change that to > README.txt to get it to process properly. > > Again, thanks. > > > Stan Klein > > > On Fri, November 11, 2011 6:00 am, Tarek Ziad? wrote: > >> >> Message: 3 >> Date: Thu, 10 Nov 2011 18:42:08 +0100 >> From: Tarek Ziad? >> To: "Stanley A. Klein" >> Cc: distutils-sig at python.org >> Subject: Re: [Distutils] Compatibility of bdist_rpm with Fedora >> ? ? ? packaging ? ? ? instructions >> >> On Thu, Nov 10, 2011 at 6:11 PM, Stanley A. Klein >> wrote: >>> Tarek - >>> >>> I downloaded pypi2rpm and built it using bdist_rpm. >> >> you built it ? pypi2rpm is a script and a bidist_rpm2 command, no need >> to build, just install >> >> >>> ?The instructions on how to use it are rather thin. >> >> Yes there's no doc, as its mostly use in our own build tools for now. >> >>> My best guess is to run it in spec-only >>> mode, modify the spec, and then run it with the modified spec specified. >>> I tried to get a help listing but that was also think and I couldn't >>> quite >>> figure out how to run it. >>> >>> Could you provide further information on how to use it. >>> >>> Thanks. >> >> Once it's installed you can build a rpm in your project, using: >> >> $ python setup.py --command-packages=pypi2rpm.command bdist_rpm2 >> --spec-file=SPECFILE >> >> (there are other options you can find with --help) >> >> >>> >>> Stan Klein >>> >>> >>> >>> On Thu, November 10, 2011 5:06 am, Tarek Ziad? wrote: >>>> On Thu, Nov 10, 2011 at 10:04 AM, Paul Nasrat >>>> wrote: >>>>> I don't think bdist_rpm should track vendor packaging requirements, >>>>> purely as those recommendations may change faster than the release >>>>> process of distutils. I also believe bdist_rpm may be going away in >>>>> the future: >>>> >>>> Yes I confirm this. We removed it in packaging because we believe it >>>> should be maintained by the RPM communities -- with their own release >>>> cycles etc. >>>> >>>> FWIW I have a custom version in the pypi2rpm project where I just feed >>>> a .spec file to the bdist_rpm command, so I can do proper RHEL or >>>> Fedora packaging. >>>> >>>>> For Fedora have you considered rpmdev-newspec which can creates a >>>>> templated python spec file for your packages. >>>>> >>>>> http://fedoraproject.org/wiki/How_to_create_an_RPM_package >>>>> >>>>> Paul >>>>> >>>>> On 8 November 2011 21:40, Stanley A. Klein wrote: >>>>>> I will need to build some Python packages for Fedora and Centos. ?The >>>>>> spec >>>>>> file produced by bdist_rpm automatically includes the statement >>>>>> %files -f INSTALLED_FILES >>>>>> >>>>>> The Fedora Python packaging instruction includes a recommendation to >>>>>> avoid >>>>>> use of INSTALLED_FILES and provides some alternatives. ?That is the >>>>>> first >>>>>> incompatibility I've encountered, but there may be more. >>>>>> >>>>>> The bdist_rpm code probably should be changed to enable >>>>>> compatibility. >>>>>> Meanwhile, is there a workaround? >>>>>> >>>>>> >>>>>> Stan Klein > > > _______________________________________________ > Distutils-SIG maillist ?- ?Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > -- Tarek Ziad? | http://ziade.org From sklein at cpcug.org Sun Nov 27 01:41:07 2011 From: sklein at cpcug.org (Stanley A. Klein) Date: Sat, 26 Nov 2011 19:41:07 -0500 (EST) Subject: [Distutils] Compatibility of bdist_rpm with Fedora packaging instructions In-Reply-To: References: <52526.71.163.68.30.1322334001.squirrel@www.cpcug.org> Message-ID: <48418.71.163.68.30.1322354467.squirrel@www.cpcug.org> Tarek - Of course. That's why I documented it. However, in a future version of bdist_rpm it would be good if some of these issues were accommodated rather than requiring manual editing and processing. Stan Klein On Sat, November 26, 2011 4:24 pm, Tarek Ziad? wrote: > Thanks for the feedback Stanley ! > > If you agree, I'll add your document into the pypi2rpm/bdist_rpm2 > project, for other people > > Cheers > Tarek > > On Sat, Nov 26, 2011 at 8:00 PM, Stanley A. Klein > wrote: >> Tarek - >> >> Thanks. >> >> I got it working and want to document some of my findings. ?I'm cc:'ing >> the Fedora Python list so they can take these issues into account in >> their >> Python packaging instructions. ?My findings are as follows: >> >> 1. ?The source code management system was git. ?I needed to install >> setuptools-git to get files recognized that were being maintained under >> git. >> >> 2. ?I also needed to establish a MANIFEST.in file to ensure all relevant >> files were included. >> >> 3. ?The setup.cfg statement under [bdist_rpm] of "doc_files =" doesn't >> work if there are directories involved. ?This is a documented "gotcha" >> in >> http://fedoraproject.org/wiki/How_to_create_an_RPM_package >> >> They also advise avoiding use of INSTALLED_FILES. >> >> Here is what I used in the spec file (it had to be edited for the >> directories to be included in docs): >> >> %install >> python setup.py install --root=$RPM_BUILD_ROOT >> cd $RPM_BUILD_ROOT >> mkdir -p %{buildroot}%{_defaultdocdir}/%{name}-%{version}/ >> mkdir -p %{buildroot}%{_defaultdocdir}/%{name}-%{version}/docs >> mkdir -p %{buildroot}%{_defaultdocdir}/%{name}-%{version}/examples >> cd ?%{_builddir}/%{name}-%{version} >> cp -p *.txt %{buildroot}%{_defaultdocdir}/%{name}-%{version}/ >> cp -rp docs/ %{buildroot}%{_defaultdocdir}/%{name}-%{version}/docs >> cp -rp examples/ >> %{buildroot}%{_defaultdocdir}/%{name}-%{version}/examples >> >> %clean >> rm -rf $RPM_BUILD_ROOT >> >> %files >> %defattr(-,root,root) >> %{_defaultdocdir}/%{name}-%{version}/ >> %{python_sitelib}/%{name}/ >> %{python_sitelib}/%{name}-%{version}-py%{python_version}.egg-info/ >> >> 4. ?The overall approach I used was to run >> >> $ python setup.py --command-packages=pypi2rpm.command bdist_rpm2 >> ?--spec-only >> >> That brought in items such as the description specified in setup.py. ?I >> then moved the spec file up a directory level from dist, edited in the >> above parts relevant to the doc files, and finally ran >> >> $ python setup.py --command-packages=pypi2rpm.command bdist_rpm2 >>> --spec-file=SPECFILE >> >> I also found it very useful to actually run >> >> python setup.py --command-packages=pypi2rpm.command bdist_rpm2 >> --spec-file >> SPECFILE ? >buildout 2>&1 >> >> Recording the build output enabled me to find details of errors that >> occurred during the build. ?There were a number of those, and the output >> flies by too quickly to notice them for diagnosis of problems that need >> to >> be fixed. ?(an approach I've also used in the past is to tee the output, >> so I can see if the build was successful when it completes). ?Also, the >> -k >> option came in useful in figuring out the documentation-related fixes to >> the spec file, because it helped seeing what was included in the build >> and >> what was available in the build but not getting copied to the install. >> >> 5. ?I inquired about how the project produced its sdist files for pypi, >> and was told that they had so many problems related to their switch to >> git >> that they simply did a gzipped tar of their repository. >> >> 6. ?The project used a different extension on their README file from the >> expected values of README and README.txt. ?I needed to change that to >> README.txt to get it to process properly. >> >> Again, thanks. >> >> >> Stan Klein >> >> >> On Fri, November 11, 2011 6:00 am, Tarek Ziad? >> wrote: >> >>> >>> Message: 3 >>> Date: Thu, 10 Nov 2011 18:42:08 +0100 >>> From: Tarek Ziad? >>> To: "Stanley A. Klein" >>> Cc: distutils-sig at python.org >>> Subject: Re: [Distutils] Compatibility of bdist_rpm with Fedora >>> ? ? ? packaging ? ? ? instructions >>> >>> On Thu, Nov 10, 2011 at 6:11 PM, Stanley A. Klein >>> wrote: >>>> Tarek - >>>> >>>> I downloaded pypi2rpm and built it using bdist_rpm. >>> >>> you built it ? pypi2rpm is a script and a bidist_rpm2 command, no need >>> to build, just install >>> >>> >>>> ?The instructions on how to use it are rather thin. >>> >>> Yes there's no doc, as its mostly use in our own build tools for now. >>> >>>> My best guess is to run it in spec-only >>>> mode, modify the spec, and then run it with the modified spec >>>> specified. >>>> I tried to get a help listing but that was also think and I couldn't >>>> quite >>>> figure out how to run it. >>>> >>>> Could you provide further information on how to use it. >>>> >>>> Thanks. >>> >>> Once it's installed you can build a rpm in your project, using: >>> >>> $ python setup.py --command-packages=pypi2rpm.command bdist_rpm2 >>> --spec-file=SPECFILE >>> >>> (there are other options you can find with --help) >>> >>> >>>> >>>> Stan Klein >>>> >>>> >>>> >>>> On Thu, November 10, 2011 5:06 am, Tarek Ziad? wrote: >>>>> On Thu, Nov 10, 2011 at 10:04 AM, Paul Nasrat >>>>> wrote: >>>>>> I don't think bdist_rpm should track vendor packaging requirements, >>>>>> purely as those recommendations may change faster than the release >>>>>> process of distutils. I also believe bdist_rpm may be going away in >>>>>> the future: >>>>> >>>>> Yes I confirm this. We removed it in packaging because we believe it >>>>> should be maintained by the RPM communities -- with their own release >>>>> cycles etc. >>>>> >>>>> FWIW I have a custom version in the pypi2rpm project where I just >>>>> feed >>>>> a .spec file to the bdist_rpm command, so I can do proper RHEL or >>>>> Fedora packaging. >>>>> >>>>>> For Fedora have you considered rpmdev-newspec which can creates a >>>>>> templated python spec file for your packages. >>>>>> >>>>>> http://fedoraproject.org/wiki/How_to_create_an_RPM_package >>>>>> >>>>>> Paul >>>>>> >>>>>> On 8 November 2011 21:40, Stanley A. Klein wrote: >>>>>>> I will need to build some Python packages for Fedora and Centos. >>>>>>> ?The >>>>>>> spec >>>>>>> file produced by bdist_rpm automatically includes the statement >>>>>>> %files -f INSTALLED_FILES >>>>>>> >>>>>>> The Fedora Python packaging instruction includes a recommendation >>>>>>> to >>>>>>> avoid >>>>>>> use of INSTALLED_FILES and provides some alternatives. ?That is the >>>>>>> first >>>>>>> incompatibility I've encountered, but there may be more. >>>>>>> >>>>>>> The bdist_rpm code probably should be changed to enable >>>>>>> compatibility. >>>>>>> Meanwhile, is there a workaround? >>>>>>> >>>>>>> >>>>>>> Stan Klein >> >> >> _______________________________________________ >> Distutils-SIG maillist ?- ?Distutils-SIG at python.org >> http://mail.python.org/mailman/listinfo/distutils-sig >> > > > > -- > Tarek Ziad? | http://ziade.org > -- From sklein at cpcug.org Sun Nov 27 18:05:08 2011 From: sklein at cpcug.org (Stanley A. Klein) Date: Sun, 27 Nov 2011 12:05:08 -0500 (EST) Subject: [Distutils] More Re: Compatibility of bdist_rpm with Fedora packaging instructions Message-ID: <59031.71.163.68.30.1322413508.squirrel@www.cpcug.org> One additional issue: The system produces pyo and pyc files for all the .py files it finds. That is good for the files that go into site-packages because they are intended to be executed from there, but might not be so good for documentation files such as examples and code-snippets that are intended to be run or otherwise used in user-space. The commands: find . -type f -name *.pyc -exec rm -f {} \; find . -type f -name *.pyo -exec rm -f {} \; executed at some point in the process at the root of the default documentation directory after the .pyc and .pyo files have been created can remove them. However, I can't seem to figure out where to put the statements. Also, might there be a way to prevent the byte compiling of documentation files? Stan Klein From sklein at cpcug.org Mon Nov 28 03:07:17 2011 From: sklein at cpcug.org (Stanley A. Klein) Date: Sun, 27 Nov 2011 21:07:17 -0500 (EST) Subject: [Distutils] More Re: Compatibility of bdist_rpm with Fedora packaging instructions Message-ID: <45546.71.163.68.30.1322446037.squirrel@www.cpcug.org> I developed a fix that can byte compile only the .py files in site-packages. It involved creating my own version of brp-python-bytecompile, preventing brp-python-bytecompile from running, and running my-python-bytecompile instead. The relevant statements in the spec file are as follows: Before the description insert (from the Fedora packaging python page): # Turn off the brp-python-bytecompile script %global __os_install_post %(echo '%{__os_install_post}' | sed -e 's!/usr/lib[^[:space:]]*/brp-python-bytecompile[[:space:]].*$!!g') At the end of the %install (following copying of the doc directories into the buildroot) insert: ~/my-rpm-macros/my-python-bytecompile The code for my-python-bytecompile is simply a copy of brp-python-bytecompile with the final section commented out. That section does files in directories other than site-packages. The code is below. Stan Klein ---------------------------------------------------------------------- my-python-bytecompile #!/bin/bash errors_terminate=$2 # If using normal root, avoid changing anything. if [ -z "$RPM_BUILD_ROOT" -o "$RPM_BUILD_ROOT" = "/" ]; then exit 0 fi # If we don't have a python interpreter, avoid changing anything. default_python=${1:-/usr/bin/python} if [ ! -x "$default_python" ]; then exit 0 fi # Figure out how deep we need to descend. We could pick an insanely high # number and hope it's enough, but somewhere, somebody's sure to run into it. depth=`(find $RPM_BUILD_ROOT -type f -name "*.py" -print0 ; echo /) | \ xargs -0 -n 1 dirname | sed 's,[^/],,g' | sort -u | tail -n 1 | wc -c` if [ -z "$depth" -o "$depth" -le "1" ]; then exit 0 fi # .pyc/.pyo files embed a "magic" value, identifying the ABI version of Python # bytecode that they are for. # # The files below RPM_BUILD_ROOT could be targetting multiple versions of # python (e.g. a single build that emits several subpackages e.g. a # python26-foo subpackage, a python31-foo subpackage etc) # # Support this by assuming that below each /usr/lib/python$VERSION/, all # .pyc/.pyo files are to be compiled for /usr/bin/python$VERSION. # # For example, below /usr/lib/python2.6/, we're targetting /usr/bin/python2.6 # and below /usr/lib/python3.1/, we're targetting /usr/bin/python3.1 shopt -s nullglob for python_libdir in $RPM_BUILD_ROOT/usr/lib{,64}/python[0-9].[0-9]/ ; do python_binary=/usr/bin/$(basename $python_libdir) real_libdir=${python_libdir/$RPM_BUILD_ROOT/} echo "Bytecompiling .py files below $python_libdir using $python_binary" # Generate normal (.pyc) byte-compiled files. $python_binary -c 'import compileall, sys; sys.exit(not compileall.compile_dir("'"$python_libdir"'", '"$depth"', "'"$real_libdir"'", force=1, quiet=1))' if [ $? -ne 0 -a 0$errors_terminate -ne 0 ]; then # One or more of the files had a syntax error exit 1 fi # Generate optimized (.pyo) byte-compiled files. $python_binary -O -c 'import compileall, sys; sys.exit(not compileall.compile_dir("'"$python_libdir"'", '"$depth"', "'"$real_libdir"'", force=1, quiet=1))' if [ $? -ne 0 -a 0$errors_terminate -ne 0 ]; then # One or more of the files had a syntax error exit 1 fi done # Handle other locations in the filesystem using the default python # implementation: # Generate normal (.pyc) byte-compiled files. #$default_python -c 'import compileall, re, sys; sys.exit (not compileall.compile_dir("'"$RPM_BUILD_ROOT"'", '"$depth"', "/", 1, re.compile(r"'"/bin/|/sbin/|/usr/lib(64)?/python[0-9]\.[0-9]"'"), quiet=1))' #if [ $? -ne 0 -a 0$errors_terminate -ne 0 ]; then # # One or more of the files had a syntax error # exit 1 #fi # # Generate optimized (.pyo) byte-compiled files. #$default_python -O -c 'import compileall, re, sys; sys.exit(not compileall.compile_dir("'"$RPM_BUILD_ROOT"'", '"$depth"', "/", 1, re.compile(r"'"/bin/|/sbin/|/usr/lib(64)?/python[0-9]\.[0-9]"'"), quiet=1))' > /dev/null #if [ $? -ne 0 -a 0$errors_terminate -ne 0 ]; then # # One or more of the files had a syntax error # exit 1 #fi exit 0