From cito at online.de Sat May 5 12:14:37 2007 From: cito at online.de (Christoph Zwerschke) Date: Sat, 05 May 2007 12:14:37 +0200 Subject: [Distutils] bdist_eggs soiling pure packages Message-ID: <463C590D.7030408@online.de> I just stumbled over the following problem, maybe someone can help me understand whether its a problem that should be fixed in setuptools or whether I'm misunderstanding something. I was packaging an application (a TurboGears quickstarted project) that has one a pure Python start script with bdist_egg. I'm using /usr/bin/env for finding the Python interpreter in the shebang line of the start script in order to make the whole thing as platform independent as possible. Now bdist_egg replaces my shebang line with the path to the python interpreter of the build platform. I just saw that this was already mentioned as a problem on the list recently: http://mail.python.org/pipermail/distutils-sig/2007-February/007306.html I agree that this should be changed as suggested there. Currently, even though an application may be completely platform independent, setuptools creates "impure" eggs by putting a platform dependent path into the scripts. I.e. I cannot package the application on one platform and easy_installing it on a different platform. I think the real problem is that setuptools replaces the shebang line when *building* the egg, not when *installing* the egg. Maybe the replacement at build time is done for backward compatibility with distutils? But then it should be done when easy_installing as well. -- Chris From jim at zope.com Sun May 6 23:19:37 2007 From: jim at zope.com (Jim Fulton) Date: Sun, 6 May 2007 17:19:37 -0400 Subject: [Distutils] Buildout bootstrap script now works when other setuptools versions are installed. Message-ID: <61FBD63C-9C3B-41F4-8A20-99A0EA0C0382@zope.com> I just fixed a bug in the zc.buildout bootstrap script: http://svn.zope.org/*checkout*/zc.buildout/trunk/bootstrap/ bootstrap.py That caused boostrapping to fail when an old setuptools was installed in the Python path (e.g. system Python). Now, the system Python is used if it's available and ez_setup is used only if necessary. Not only does this avoid failures when older versions of setuptools is installed, but also makes bootstrapping a little faster when setuptools is installed locally. People including bootstrap.py in their projects should update to the new version. Jim -- Jim Fulton mailto:jim at zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From b.n.lawrence at rl.ac.uk Tue May 8 17:27:17 2007 From: b.n.lawrence at rl.ac.uk (Bryan Lawrence) Date: Tue, 8 May 2007 16:27:17 +0100 Subject: [Distutils] easy_install and sys.path Message-ID: <200705081627.17538.b.n.lawrence@rl.ac.uk> Hi Folks Can anyone point me to why easy_install forces modules to the front of sys.path? It seems to break long established rules about ordering of path variables, and certainly breaks python in some unexpected contexts e.g: https://bugs.launchpad.net/ubuntu/+source/kubuntu-meta/+bug/113298 Thanks in advance, Bryan From tseaver at palladion.com Tue May 8 18:20:45 2007 From: tseaver at palladion.com (Tres Seaver) Date: Tue, 08 May 2007 12:20:45 -0400 Subject: [Distutils] easy_install and sys.path In-Reply-To: <200705081627.17538.b.n.lawrence@rl.ac.uk> References: <200705081627.17538.b.n.lawrence@rl.ac.uk> Message-ID: <4640A35D.3010002@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Bryan Lawrence wrote: > Can anyone point me to why easy_install forces modules to the front > of sys.path? It seems to break long established rules about ordering > of path variables, and certainly breaks python in some unexpected > contexts e.g: > https://bugs.launchpad.net/ubuntu/+source/kubuntu-meta/+bug/113298 In your case, you have libraries mixed from two different version s of python. From that launchpad issue: > ok, and quite clearly my path has all the /usr/local stuff in it first: >>>> import sys >>>> print sys.path > ['', '/usr/local/lib/python2.5/site-packages/simplejson-1.4-py2.5.egg', '/usr/local/lib/python2.5/site-packages/Amara-1.1.9-py2.5.egg', '/usr/local/lib/python2.5/site-packages/flup-0.5-py2.5.egg', '/usr/local/lib/python2.5/site-packages/scgi-1.12-py2.5-linux-i686.egg', '/usr/local/lib/python2.5/site-packages/Paste-1.0-py2.5.egg', '/usr/local/lib/python2.5/site-packages/ZSI-2.0.dev_r1293-py2.5.egg', '/usr/local/lib/python2.5/site-packages/dap-2.2.5.1-py2.5.egg', '/usr/local/lib/python2.5/site-packages/httplib2-0.2.0-py2.5.egg', '/usr/local/lib/python2.5/site-packages/Cheetah-2.0rc7-py2.5-linux-i686.egg', '/usr/local/lib/python2.5/site-packages/dap.plugins.grads-0.1.1-py2.5.egg', '/usr/local/lib/python2.5/site-packages/arrayterator-0.2.5-py2.5.egg', '/usr/local/lib/python2.5/site-packages/PasteDeploy-1.0-py2.5.egg', '/usr/local/lib/python2.5/site-packages/PasteScript-1.0-py2.5.egg', '/usr/local/lib/python2.5/site-packages/dap.plugins.netcdf-0.3.2-py2.5.egg', '/usr/local/lib/python2.5/site-packages/pupynere-0.2.1-py2.5.egg', '/usr/local/lib/python2.5/site-packages/wsgistate-0.4-py2.5.egg', '/usr/local/lib/python2.5/site-packages/SQLAlchemy-0.3.3-py2.5.egg', '/usr/local/lib/python2.5/site-packages/WSGIUtils-0.7-py2.5.egg', '/usr/local/lib/python2.5/site-packages/setuptools-0.6c5-py2.5.egg', '/usr/local/lib/python2.5/site-packages/kid-0.9.5-py2.5.egg', '/usr/local/lib/python2.5/site-packages/PyOpenGL-3.0.0a5-py2.5.egg', '/usr/local/lib/python2.5/site-packages/Genshi-0.4-py2.5.egg', '/usr/local/lib/python2.5/site-packages/wsgilog-0.1-py2.5.egg', '/usr/lib/python25.zip', '/usr/lib/python2.5', '/usr/lib/python2.5/plat-linux2', '/usr/lib/python2.5/lib-tk', '/usr/lib/python2.5/lib-dynload', '/usr/local/lib/python2.5/site-packages', '/usr/lib/python2.5/site-packages', '/var/lib/python-support/python2.5'] AFAIK, This can only happen because you have a .pth file in some 'site' directory which is mistakenly pulling in the "alien" eggs, which should never happen. *None* of those eggs are guaranteed to work in a different Python than the one which built them; in your case, the UCS2 vs. UCS4 bit is interfering, but it could equally well be a different dynload problem. If you run with 'site' stuff disabled, is your sys path free of those eggs? E.g.: $ /usr/bin/python -S If so, find the errant .pth file and remove it (or fix the paths in it). Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGQKNd+gerLs4ltQ4RAuMjAKCluKWopcuCk/rzaIRXdKF4tOqiywCfWauk GAuW/hxxNkIPnEbRGlu6f08= =ya5z -----END PGP SIGNATURE----- From tseaver at palladion.com Tue May 8 18:20:45 2007 From: tseaver at palladion.com (Tres Seaver) Date: Tue, 08 May 2007 12:20:45 -0400 Subject: [Distutils] easy_install and sys.path In-Reply-To: <200705081627.17538.b.n.lawrence@rl.ac.uk> References: <200705081627.17538.b.n.lawrence@rl.ac.uk> Message-ID: <4640A35D.3010002@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Bryan Lawrence wrote: > Can anyone point me to why easy_install forces modules to the front > of sys.path? It seems to break long established rules about ordering > of path variables, and certainly breaks python in some unexpected > contexts e.g: > https://bugs.launchpad.net/ubuntu/+source/kubuntu-meta/+bug/113298 In your case, you have libraries mixed from two different version s of python. From that launchpad issue: > ok, and quite clearly my path has all the /usr/local stuff in it first: >>>> import sys >>>> print sys.path > ['', '/usr/local/lib/python2.5/site-packages/simplejson-1.4-py2.5.egg', '/usr/local/lib/python2.5/site-packages/Amara-1.1.9-py2.5.egg', '/usr/local/lib/python2.5/site-packages/flup-0.5-py2.5.egg', '/usr/local/lib/python2.5/site-packages/scgi-1.12-py2.5-linux-i686.egg', '/usr/local/lib/python2.5/site-packages/Paste-1.0-py2.5.egg', '/usr/local/lib/python2.5/site-packages/ZSI-2.0.dev_r1293-py2.5.egg', '/usr/local/lib/python2.5/site-packages/dap-2.2.5.1-py2.5.egg', '/usr/local/lib/python2.5/site-packages/httplib2-0.2.0-py2.5.egg', '/usr/local/lib/python2.5/site-packages/Cheetah-2.0rc7-py2.5-linux-i686.egg', '/usr/local/lib/python2.5/site-packages/dap.plugins.grads-0.1.1-py2.5.egg', '/usr/local/lib/python2.5/site-packages/arrayterator-0.2.5-py2.5.egg', '/usr/local/lib/python2.5/site-packages/PasteDeploy-1.0-py2.5.egg', '/usr/local/lib/python2.5/site-packages/PasteScript-1.0-py2.5.egg', '/usr/local/lib/python2.5/site-packages/dap.plugins.netcdf-0.3.2-py2.5.egg', '/usr/local/lib/python2.5/site-packages/pupynere-0.2.1-py2.5.egg', '/usr/local/lib/python2.5/site-packages/wsgistate-0.4-py2.5.egg', '/usr/local/lib/python2.5/site-packages/SQLAlchemy-0.3.3-py2.5.egg', '/usr/local/lib/python2.5/site-packages/WSGIUtils-0.7-py2.5.egg', '/usr/local/lib/python2.5/site-packages/setuptools-0.6c5-py2.5.egg', '/usr/local/lib/python2.5/site-packages/kid-0.9.5-py2.5.egg', '/usr/local/lib/python2.5/site-packages/PyOpenGL-3.0.0a5-py2.5.egg', '/usr/local/lib/python2.5/site-packages/Genshi-0.4-py2.5.egg', '/usr/local/lib/python2.5/site-packages/wsgilog-0.1-py2.5.egg', '/usr/lib/python25.zip', '/usr/lib/python2.5', '/usr/lib/python2.5/plat-linux2', '/usr/lib/python2.5/lib-tk', '/usr/lib/python2.5/lib-dynload', '/usr/local/lib/python2.5/site-packages', '/usr/lib/python2.5/site-packages', '/var/lib/python-support/python2.5'] AFAIK, This can only happen because you have a .pth file in some 'site' directory which is mistakenly pulling in the "alien" eggs, which should never happen. *None* of those eggs are guaranteed to work in a different Python than the one which built them; in your case, the UCS2 vs. UCS4 bit is interfering, but it could equally well be a different dynload problem. If you run with 'site' stuff disabled, is your sys path free of those eggs? E.g.: $ /usr/bin/python -S If so, find the errant .pth file and remove it (or fix the paths in it). Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGQKNd+gerLs4ltQ4RAuMjAKCluKWopcuCk/rzaIRXdKF4tOqiywCfWauk GAuW/hxxNkIPnEbRGlu6f08= =ya5z -----END PGP SIGNATURE----- From b.n.lawrence at rl.ac.uk Wed May 9 09:25:39 2007 From: b.n.lawrence at rl.ac.uk (Bryan Lawrence) Date: Wed, 9 May 2007 08:25:39 +0100 Subject: [Distutils] easy_install and sys.path In-Reply-To: <4640A35D.3010002@palladion.com> References: <200705081627.17538.b.n.lawrence@rl.ac.uk> <4640A35D.3010002@palladion.com> Message-ID: <200705090825.39865.b.n.lawrence@rl.ac.uk> Hi Tres Oh I know I shouldn't mix two pythons, but I've never had them "mixed" before. (We have run our own python in /usr/local for yonks without any bad karma until now). It's only the advent of eggs, which push things to the top of sys.path that causes a problem. It would appear that I'm not the only person who has the problem (e.g. see http://www.rexx.com/~dkuhlman/python_comments.html). So, my question to this list is "why"? Why does easy_install put things in the front of sys.path? Presumably there is some good reason (perhaps it is the only way to ensure that your dependency libraries are the ones that are picked up, I guess there is a risk that older libraries might exist ... but it causes problems for the reasons that Dave Kuhlman lists in his page linked above ). It would appear that there is fairly long established python behaviour that "new" libraries go after "old" libraries, which easy_install has broken. At this stage, all I want to do is understand why easy_install chose to do that. I've been a big fan of eggs, but I've just slammed into a wall. I can't believe that I'm the only one for whom this may be a problem. I now need to work out what, if anything, I need to do about it (change my behaviour, find out something I didn't know about egg installation, provide a suggested optional change to egg behaviour, go home have a drink etc ...) Cheers Bryan On Tuesday 08 May 2007 17:20:45 Tres Seaver wrote: > Bryan Lawrence wrote: > > Can anyone point me to why easy_install forces modules to the front > > of sys.path? It seems to break long established rules about ordering > > of path variables, and certainly breaks python in some unexpected > > contexts e.g: > > https://bugs.launchpad.net/ubuntu/+source/kubuntu-meta/+bug/113298 > > In your case, you have libraries mixed from two different version s of > > python. From that launchpad issue: > > ok, and quite clearly my path has all the /usr/local stuff in it first: > >>>> import sys > >>>> print sys.path > > > > ['', > > '/usr/local/lib/python2.5/site-packages/simplejson-1.4-py2.5.egg', > '/usr/local/lib/python2.5/site-packages/Amara-1.1.9-py2.5.egg', > '/usr/local/lib/python2.5/site-packages/flup-0.5-py2.5.egg', > '/usr/local/lib/python2.5/site-packages/scgi-1.12-py2.5-linux-i686.egg', > '/usr/local/lib/python2.5/site-packages/Paste-1.0-py2.5.egg', > '/usr/local/lib/python2.5/site-packages/ZSI-2.0.dev_r1293-py2.5.egg', > '/usr/local/lib/python2.5/site-packages/dap-2.2.5.1-py2.5.egg', > '/usr/local/lib/python2.5/site-packages/httplib2-0.2.0-py2.5.egg', > '/usr/local/lib/python2.5/site-packages/Cheetah-2.0rc7-py2.5-linux-i686.egg >', > '/usr/local/lib/python2.5/site-packages/dap.plugins.grads-0.1.1-py2.5.egg', > '/usr/local/lib/python2.5/site-packages/arrayterator-0.2.5-py2.5.egg', > '/usr/local/lib/python2.5/site-packages/PasteDeploy-1.0-py2.5.egg', > '/usr/local/lib/python2.5/site-packages/PasteScript-1.0-py2.5.egg', > '/usr/local/lib/python2.5/site-packages/dap.plugins.netcdf-0.3.2-py2.5.egg' >, '/usr/local/lib/python2.5/site-packages/pupynere-0.2.1-py2.5.egg', > '/usr/local/lib/python2.5/site-packages/wsgistate-0.4-py2.5.egg', > '/usr/local/lib/python2.5/site-packages/SQLAlchemy-0.3.3-py2.5.egg', > '/usr/local/lib/python2.5/site-packages/WSGIUtils-0.7-py2.5.egg', > '/usr/local/lib/python2.5/site-packages/setuptools-0.6c5-py2.5.egg', > '/usr/local/lib/python2.5/site-packages/kid-0.9.5-py2.5.egg', > '/usr/local/lib/python2.5/site-packages/PyOpenGL-3.0.0a5-py2.5.egg', > '/usr/local/lib/python2.5/site-packages/Genshi-0.4-py2.5.egg', > '/usr/local/lib/python2.5/site-packages/wsgilog-0.1-py2.5.egg', > '/usr/lib/python25.zip', '/usr/lib/python2.5', > '/usr/lib/python2.5/plat-linux2', '/usr/lib/python2.5/lib-tk', > '/usr/lib/python2.5/lib-dynload', > '/usr/local/lib/python2.5/site-packages', > '/usr/lib/python2.5/site-packages', '/var/lib/python-support/python2.5'] > > AFAIK, This can only happen because you have a .pth file in some 'site' > directory which is mistakenly pulling in the "alien" eggs, which should > never happen. *None* of those eggs are guaranteed to work in a > different Python than the one which built them; in your case, the UCS2 > vs. UCS4 bit is interfering, but it could equally well be a different > dynload problem. > > If you run with 'site' stuff disabled, is your sys path free of those > eggs? E.g.: > > $ /usr/bin/python -S > > If so, find the errant .pth file and remove it (or fix the paths in it). > > > Tres. From matt at matt-good.net Wed May 9 09:52:45 2007 From: matt at matt-good.net (Matt Good) Date: Wed, 09 May 2007 00:52:45 -0700 Subject: [Distutils] easy_install and sys.path In-Reply-To: <200705090825.39865.b.n.lawrence@rl.ac.uk> References: <200705081627.17538.b.n.lawrence@rl.ac.uk> <4640A35D.3010002@palladion.com> <200705090825.39865.b.n.lawrence@rl.ac.uk> Message-ID: <46417DCD.6070607@matt-good.net> You seem to have missed the main point of Tres's email. The problem is not inherent to Eggs, but you seem to have either set PYTHONPATH or have a .pth file that's being loaded which is telling your /usr Python to look in the /usr/local Python's site-packages folder. Packages installed to /usr/local in the traditional way would still be on sys.path and would have the exact same problem. So check your PYTHONPATH and grep your .pth files in /usr/lib/python2.5/site-packages for any references to /usr/local and remove them. -- Matt Good Bryan Lawrence wrote: > Hi Tres > > Oh I know I shouldn't mix two pythons, but I've never had them "mixed" before. > (We have run our own python in /usr/local for yonks without any bad karma > until now). It's only the advent of eggs, which push things to the top of > sys.path that causes a problem. It would appear that I'm not the only person > who has the problem (e.g. see > http://www.rexx.com/~dkuhlman/python_comments.html). > > So, my question to this list is "why"? > > Why does easy_install put things in the front of sys.path? Presumably there is > some good reason (perhaps it is the only way to ensure that your dependency > libraries are the ones that are picked up, I guess there is a risk that older > libraries might exist ... but it causes problems for the reasons that Dave > Kuhlman lists in his page linked above ). > > It would appear that there is fairly long established python behaviour > that "new" libraries go after "old" libraries, which easy_install has broken. > At this stage, all I want to do is understand why easy_install chose to do > that. > > I've been a big fan of eggs, but I've just slammed into a wall. I can't > believe that I'm the only one for whom this may be a problem. I now need to > work out what, if anything, I need to do about it (change my behaviour, find > out something I didn't know about egg installation, provide a suggested > optional change to egg behaviour, go home have a drink etc ...) > > Cheers > Bryan > > On Tuesday 08 May 2007 17:20:45 Tres Seaver wrote: > >> Bryan Lawrence wrote: >> >>> Can anyone point me to why easy_install forces modules to the front >>> of sys.path? It seems to break long established rules about ordering >>> of path variables, and certainly breaks python in some unexpected >>> contexts e.g: >>> https://bugs.launchpad.net/ubuntu/+source/kubuntu-meta/+bug/113298 >>> >> In your case, you have libraries mixed from two different version s of >> >> python. From that launchpad issue: >> >>> ok, and quite clearly my path has all the /usr/local stuff in it first: >>> >>>>>> import sys >>>>>> print sys.path >>>>>> >>> ['', >>> >> '/usr/local/lib/python2.5/site-packages/simplejson-1.4-py2.5.egg', >> '/usr/local/lib/python2.5/site-packages/Amara-1.1.9-py2.5.egg', >> '/usr/local/lib/python2.5/site-packages/flup-0.5-py2.5.egg', >> '/usr/local/lib/python2.5/site-packages/scgi-1.12-py2.5-linux-i686.egg', >> '/usr/local/lib/python2.5/site-packages/Paste-1.0-py2.5.egg', >> '/usr/local/lib/python2.5/site-packages/ZSI-2.0.dev_r1293-py2.5.egg', >> '/usr/local/lib/python2.5/site-packages/dap-2.2.5.1-py2.5.egg', >> '/usr/local/lib/python2.5/site-packages/httplib2-0.2.0-py2.5.egg', >> '/usr/local/lib/python2.5/site-packages/Cheetah-2.0rc7-py2.5-linux-i686.egg >> ', >> '/usr/local/lib/python2.5/site-packages/dap.plugins.grads-0.1.1-py2.5.egg', >> '/usr/local/lib/python2.5/site-packages/arrayterator-0.2.5-py2.5.egg', >> '/usr/local/lib/python2.5/site-packages/PasteDeploy-1.0-py2.5.egg', >> '/usr/local/lib/python2.5/site-packages/PasteScript-1.0-py2.5.egg', >> '/usr/local/lib/python2.5/site-packages/dap.plugins.netcdf-0.3.2-py2.5.egg' >> , '/usr/local/lib/python2.5/site-packages/pupynere-0.2.1-py2.5.egg', >> '/usr/local/lib/python2.5/site-packages/wsgistate-0.4-py2.5.egg', >> '/usr/local/lib/python2.5/site-packages/SQLAlchemy-0.3.3-py2.5.egg', >> '/usr/local/lib/python2.5/site-packages/WSGIUtils-0.7-py2.5.egg', >> '/usr/local/lib/python2.5/site-packages/setuptools-0.6c5-py2.5.egg', >> '/usr/local/lib/python2.5/site-packages/kid-0.9.5-py2.5.egg', >> '/usr/local/lib/python2.5/site-packages/PyOpenGL-3.0.0a5-py2.5.egg', >> '/usr/local/lib/python2.5/site-packages/Genshi-0.4-py2.5.egg', >> '/usr/local/lib/python2.5/site-packages/wsgilog-0.1-py2.5.egg', >> '/usr/lib/python25.zip', '/usr/lib/python2.5', >> '/usr/lib/python2.5/plat-linux2', '/usr/lib/python2.5/lib-tk', >> '/usr/lib/python2.5/lib-dynload', >> '/usr/local/lib/python2.5/site-packages', >> '/usr/lib/python2.5/site-packages', '/var/lib/python-support/python2.5'] >> >> AFAIK, This can only happen because you have a .pth file in some 'site' >> directory which is mistakenly pulling in the "alien" eggs, which should >> never happen. *None* of those eggs are guaranteed to work in a >> different Python than the one which built them; in your case, the UCS2 >> vs. UCS4 bit is interfering, but it could equally well be a different >> dynload problem. >> >> If you run with 'site' stuff disabled, is your sys path free of those >> eggs? E.g.: >> >> $ /usr/bin/python -S >> >> If so, find the errant .pth file and remove it (or fix the paths in it). >> >> >> Tres. >> > > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > From b.n.lawrence at rl.ac.uk Wed May 9 11:00:06 2007 From: b.n.lawrence at rl.ac.uk (Bryan Lawrence) Date: Wed, 9 May 2007 10:00:06 +0100 Subject: [Distutils] easy_install and sys.path In-Reply-To: <46417DCD.6070607@matt-good.net> References: <200705081627.17538.b.n.lawrence@rl.ac.uk> <200705090825.39865.b.n.lawrence@rl.ac.uk> <46417DCD.6070607@matt-good.net> Message-ID: <200705091000.06482.b.n.lawrence@rl.ac.uk> On Wednesday 09 May 2007 08:52:45 Matt Good wrote: > You seem to have missed the main point of Tres's email. Wouldn't be the first time I missed the point, and wont be the last :-) Thanks to you both. > The problem is > not inherent to Eggs, but you seem to have either set PYTHONPATH No. > or have > a .pth file that's being loaded which is telling your /usr Python to > look in the /usr/local Python's site-packages folder. The /usr python doesn't have such a .pth file*, but there is one in the /usr/local/lib/python2.5/site-packages, which seems to be inserting the eggs. * the only one that exists in the /usr hierarchy is /usr/lib/python2.5/site-packages/python-support.pth which points to /var/lib/python-support/python2.5 > Packages > installed to /usr/local in the traditional way would still be on > sys.path and would have the exact same problem. No I don't think so, because they should be in sys.path after the packages in /usr/lib/, the problem is manifesting here because the eggs are inserted in the top of the site-path. (But as tres points out, any number of other dynamic load problems could manifest themselves next). > So check your > PYTHONPATH and grep your .pth files in /usr/lib/python2.5/site-packages > for any references to /usr/local and remove them. So now I'm twice as confused. 1) I don't know why the .pth file in my /usr/local/site-packages is being processed by my /usr python 2) I don't know why eggs want to be first in the queue. Cheers Bryan From bhendrix at enthought.com Wed May 9 17:30:11 2007 From: bhendrix at enthought.com (Bryce Hendrix) Date: Wed, 09 May 2007 10:30:11 -0500 Subject: [Distutils] easy_install and sys.path In-Reply-To: <46417DCD.6070607@matt-good.net> References: <200705081627.17538.b.n.lawrence@rl.ac.uk> <4640A35D.3010002@palladion.com> <200705090825.39865.b.n.lawrence@rl.ac.uk> <46417DCD.6070607@matt-good.net> Message-ID: <4641E903.9090402@enthought.com> Bryan still raises a valid question. The fact that setuptools shoves the eggs to the front of sys.path continues to cause false bug reports to be filed by our customers also regularly. Bryce Matt Good wrote: > You seem to have missed the main point of Tres's email. The problem is > not inherent to Eggs, but you seem to have either set PYTHONPATH or have > a .pth file that's being loaded which is telling your /usr Python to > look in the /usr/local Python's site-packages folder. Packages > installed to /usr/local in the traditional way would still be on > sys.path and would have the exact same problem. So check your > PYTHONPATH and grep your .pth files in /usr/lib/python2.5/site-packages > for any references to /usr/local and remove them. > > -- Matt Good > > Bryan Lawrence wrote: > >> Hi Tres >> >> Oh I know I shouldn't mix two pythons, but I've never had them "mixed" before. >> (We have run our own python in /usr/local for yonks without any bad karma >> until now). It's only the advent of eggs, which push things to the top of >> sys.path that causes a problem. It would appear that I'm not the only person >> who has the problem (e.g. see >> http://www.rexx.com/~dkuhlman/python_comments.html). >> >> So, my question to this list is "why"? >> >> Why does easy_install put things in the front of sys.path? Presumably there is >> some good reason (perhaps it is the only way to ensure that your dependency >> libraries are the ones that are picked up, I guess there is a risk that older >> libraries might exist ... but it causes problems for the reasons that Dave >> Kuhlman lists in his page linked above ). >> >> It would appear that there is fairly long established python behaviour >> that "new" libraries go after "old" libraries, which easy_install has broken. >> At this stage, all I want to do is understand why easy_install chose to do >> that. >> >> I've been a big fan of eggs, but I've just slammed into a wall. I can't >> believe that I'm the only one for whom this may be a problem. I now need to >> work out what, if anything, I need to do about it (change my behaviour, find >> out something I didn't know about egg installation, provide a suggested >> optional change to egg behaviour, go home have a drink etc ...) >> >> Cheers >> Bryan >> >> On Tuesday 08 May 2007 17:20:45 Tres Seaver wrote: >> >> >>> Bryan Lawrence wrote: >>> >>> >>>> Can anyone point me to why easy_install forces modules to the front >>>> of sys.path? It seems to break long established rules about ordering >>>> of path variables, and certainly breaks python in some unexpected >>>> contexts e.g: >>>> https://bugs.launchpad.net/ubuntu/+source/kubuntu-meta/+bug/113298 >>>> >>>> >>> In your case, you have libraries mixed from two different version s of >>> >>> python. From that launchpad issue: >>> >>> >>>> ok, and quite clearly my path has all the /usr/local stuff in it first: >>>> >>>> >>>>>>> import sys >>>>>>> print sys.path >>>>>>> >>>>>>> >>>> ['', >>>> >>>> >>> '/usr/local/lib/python2.5/site-packages/simplejson-1.4-py2.5.egg', >>> '/usr/local/lib/python2.5/site-packages/Amara-1.1.9-py2.5.egg', >>> '/usr/local/lib/python2.5/site-packages/flup-0.5-py2.5.egg', >>> '/usr/local/lib/python2.5/site-packages/scgi-1.12-py2.5-linux-i686.egg', >>> '/usr/local/lib/python2.5/site-packages/Paste-1.0-py2.5.egg', >>> '/usr/local/lib/python2.5/site-packages/ZSI-2.0.dev_r1293-py2.5.egg', >>> '/usr/local/lib/python2.5/site-packages/dap-2.2.5.1-py2.5.egg', >>> '/usr/local/lib/python2.5/site-packages/httplib2-0.2.0-py2.5.egg', >>> '/usr/local/lib/python2.5/site-packages/Cheetah-2.0rc7-py2.5-linux-i686.egg >>> ', >>> '/usr/local/lib/python2.5/site-packages/dap.plugins.grads-0.1.1-py2.5.egg', >>> '/usr/local/lib/python2.5/site-packages/arrayterator-0.2.5-py2.5.egg', >>> '/usr/local/lib/python2.5/site-packages/PasteDeploy-1.0-py2.5.egg', >>> '/usr/local/lib/python2.5/site-packages/PasteScript-1.0-py2.5.egg', >>> '/usr/local/lib/python2.5/site-packages/dap.plugins.netcdf-0.3.2-py2.5.egg' >>> , '/usr/local/lib/python2.5/site-packages/pupynere-0.2.1-py2.5.egg', >>> '/usr/local/lib/python2.5/site-packages/wsgistate-0.4-py2.5.egg', >>> '/usr/local/lib/python2.5/site-packages/SQLAlchemy-0.3.3-py2.5.egg', >>> '/usr/local/lib/python2.5/site-packages/WSGIUtils-0.7-py2.5.egg', >>> '/usr/local/lib/python2.5/site-packages/setuptools-0.6c5-py2.5.egg', >>> '/usr/local/lib/python2.5/site-packages/kid-0.9.5-py2.5.egg', >>> '/usr/local/lib/python2.5/site-packages/PyOpenGL-3.0.0a5-py2.5.egg', >>> '/usr/local/lib/python2.5/site-packages/Genshi-0.4-py2.5.egg', >>> '/usr/local/lib/python2.5/site-packages/wsgilog-0.1-py2.5.egg', >>> '/usr/lib/python25.zip', '/usr/lib/python2.5', >>> '/usr/lib/python2.5/plat-linux2', '/usr/lib/python2.5/lib-tk', >>> '/usr/lib/python2.5/lib-dynload', >>> '/usr/local/lib/python2.5/site-packages', >>> '/usr/lib/python2.5/site-packages', '/var/lib/python-support/python2.5'] >>> >>> AFAIK, This can only happen because you have a .pth file in some 'site' >>> directory which is mistakenly pulling in the "alien" eggs, which should >>> never happen. *None* of those eggs are guaranteed to work in a >>> different Python than the one which built them; in your case, the UCS2 >>> vs. UCS4 bit is interfering, but it could equally well be a different >>> dynload problem. >>> >>> If you run with 'site' stuff disabled, is your sys path free of those >>> eggs? E.g.: >>> >>> $ /usr/bin/python -S >>> >>> If so, find the errant .pth file and remove it (or fix the paths in it). >>> >>> >>> Tres. >>> >>> >> _______________________________________________ >> Distutils-SIG maillist - Distutils-SIG at python.org >> http://mail.python.org/mailman/listinfo/distutils-sig >> >> > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20070509/f4d8d4d1/attachment.html From pje at telecommunity.com Wed May 9 19:41:52 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 09 May 2007 13:41:52 -0400 Subject: [Distutils] easy_install and sys.path In-Reply-To: <200705090825.39865.b.n.lawrence@rl.ac.uk> References: <200705081627.17538.b.n.lawrence@rl.ac.uk> <4640A35D.3010002@palladion.com> <200705090825.39865.b.n.lawrence@rl.ac.uk> Message-ID: <20070509174007.A81D13A409D@sparrow.telecommunity.com> At 08:25 AM 5/9/2007 +0100, Bryan Lawrence wrote: >Oh I know I shouldn't mix two pythons, but I've never had them "mixed" before. >(We have run our own python in /usr/local for yonks without any bad karma >until now). It's only the advent of eggs, which push things to the top of >sys.path that causes a problem. It would appear that I'm not the only person >who has the problem (e.g. see >http://www.rexx.com/~dkuhlman/python_comments.html). > >So, my question to this list is "why"? > >Why does easy_install put things in the front of sys.path? >Presumably there is >some good reason (perhaps it is the only way to ensure that your dependency >libraries are the ones that are picked up, I guess there is a risk that older >libraries might exist That's precisely the reason. Until it started doing this, it had to try to either delete those conflicting libraries, or refuse to install anything until the user fixed it. >I've been a big fan of eggs, but I've just slammed into a wall. I can't >believe that I'm the only one for whom this may be a problem. I now need to >work out what, if anything, I need to do about it (change my behaviour, find >out something I didn't know about egg installation, provide a suggested >optional change to egg behaviour, go home have a drink etc ...) You can use the "-m" option to easy_install, in which case eggs will only be added to sys.path when you ask for them. Just delete your existing easy-install.pth file (leaving the setuptools.pth file in place). Then, your existing eggs will still be there, and any easy_install-provided scripts will still work. You just will have to manually require() any eggs you want when you run any program that's not a setuptools-generated script, or when you fire up the interpreter (unless you modify your .pythonrc to require() the eggs you want). > > '/usr/local/lib/python2.5/site-packages', The fact that this is in your sys.path is what's causing those eggs to be added, btw. It appears your distro may have either patched /usr/lib/python2.5/site.py to include the /usr/local tree, or you have a .pth file in /usr/lib/python2.5/site-packages that adds the /usr/local/.../site-packages directory. From pje at telecommunity.com Wed May 9 20:26:01 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 09 May 2007 14:26:01 -0400 Subject: [Distutils] bdist_eggs soiling pure packages In-Reply-To: <463C590D.7030408@online.de> References: <463C590D.7030408@online.de> Message-ID: <20070509182418.3CF133A4061@sparrow.telecommunity.com> At 12:14 PM 5/5/2007 +0200, Christoph Zwerschke wrote: >I just stumbled over the following problem, maybe someone can help me >understand whether its a problem that should be fixed in setuptools or >whether I'm misunderstanding something. > >I was packaging an application (a TurboGears quickstarted project) that >has one a pure Python start script with bdist_egg. I'm using >/usr/bin/env for finding the Python interpreter in the shebang line of >the start script in order to make the whole thing as platform >independent as possible. > >Now bdist_egg replaces my shebang line with the path to the python >interpreter of the build platform. I just saw that this was already >mentioned as a problem on the list recently: >http://mail.python.org/pipermail/distutils-sig/2007-February/007306.html >I agree that this should be changed as suggested there. > >Currently, even though an application may be completely platform >independent, setuptools creates "impure" eggs by putting a platform >dependent path into the scripts. I.e. I cannot package the application >on one platform and easy_installing it on a different platform. > >I think the real problem is that setuptools replaces the shebang line >when *building* the egg, not when *installing* the egg. Use entry point scripts instead; they don't have this problem: http://peak.telecommunity.com/DevCenter/setuptools#automatic-script-creation From torfer at pvv.org Wed May 9 22:44:51 2007 From: torfer at pvv.org (Tore Ferner) Date: Wed, 09 May 2007 22:44:51 +0200 Subject: [Distutils] stdeb bug: syntax error during dpkg -i Message-ID: <464232C3.7030300@pvv.org> Hi, stdeb seems like a good idea... I tried to use 0.2.a1 on bzr http://bazaar-vcs.org/releases/src/bzr-0.16.tar.gz and got the error message below. I run Debian unstable; I've paused upgrading since just after Etch was released. At the end you can see my installed python modules. On my system python equals python2.4, but for some things python2.3 is used (for some reason unknown to me, but it has worked fine so far). Generated thus: python -c "import stdeb; execfile('setup.py')" sdist_dsc cd deb_dist/bzr-0.16.0/ dpkg-buildpackage -tc dpkg -i ../python-bzr_0.16.0-1_all.deb Best regards Tore (Reading database ... 242979 files and directories currently installed.) Preparing to replace python-bzr 0.16.0-1 (using python-bzr_0.16.0-1_all.deb) ... INFO: using old version '/usr/bin/python2.3' INFO: using unsupported version '/usr/bin/python2.5' Unpacking replacement python-bzr ... Setting up python-bzr (0.16.0-1) ... Compiling /usr/lib/python2.3/site-packages/bzrlib/benchmarks/tree_creator/__init__.py ... File "/usr/lib/python2.3/site-packages/bzrlib/benchmarks/tree_creator/__init__.py", line 25 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/benchmarks/tree_creator/heavily_merged.py ... File "/usr/lib/python2.3/site-packages/bzrlib/benchmarks/tree_creator/heavily_merged.py", line 22 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/benchmarks/tree_creator/kernel_like.py ... File "/usr/lib/python2.3/site-packages/bzrlib/benchmarks/tree_creator/kernel_like.py", line 22 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/benchmarks/tree_creator/many_commit.py ... File "/usr/lib/python2.3/site-packages/bzrlib/benchmarks/tree_creator/many_commit.py", line 19 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/benchmarks/tree_creator/simple_many_commit.py ... File "/usr/lib/python2.3/site-packages/bzrlib/benchmarks/tree_creator/simple_many_commit.py", line 19 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/benchmarks/__init__.py ... File "/usr/lib/python2.3/site-packages/bzrlib/benchmarks/__init__.py", line 19 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/benchmarks/bench_bench.py ... File "/usr/lib/python2.3/site-packages/bzrlib/benchmarks/bench_bench.py", line 19 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/benchmarks/bench_cache_utf8.py ... File "/usr/lib/python2.3/site-packages/bzrlib/benchmarks/bench_cache_utf8.py", line 20 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/benchmarks/bench_sftp.py ... File "/usr/lib/python2.3/site-packages/bzrlib/benchmarks/bench_sftp.py", line 21 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/benchmarks/bench_transform.py ... File "/usr/lib/python2.3/site-packages/bzrlib/benchmarks/bench_transform.py", line 33 paths.extend(pathjoin(dirpath, d) for d in dirnames) ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/benchmarks/bench_xml.py ... File "/usr/lib/python2.3/site-packages/bzrlib/benchmarks/bench_xml.py", line 19 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/bundle/serializer/v08.py ... File "/usr/lib/python2.3/site-packages/bzrlib/bundle/serializer/v08.py", line 23 from bzrlib.bundle.serializer import (BundleSerializer, ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/bundle/bundle_data.py ... File "/usr/lib/python2.3/site-packages/bzrlib/bundle/bundle_data.py", line 24 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/export/zip_exporter.py ... File "/usr/lib/python2.3/site-packages/bzrlib/export/zip_exporter.py", line 24 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/plugins/launchpad/__init__.py ... File "/usr/lib/python2.3/site-packages/bzrlib/plugins/launchpad/__init__.py", line 87 from lp_registration import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/plugins/launchpad/lp_indirect.py ... File "/usr/lib/python2.3/site-packages/bzrlib/plugins/launchpad/lp_indirect.py", line 25 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/plugins/launchpad/lp_registration.py ... File "/usr/lib/python2.3/site-packages/bzrlib/plugins/launchpad/lp_registration.py", line 60 @property ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/plugins/launchpad/test_lp_indirect.py ... File "/usr/lib/python2.3/site-packages/bzrlib/plugins/launchpad/test_lp_indirect.py", line 19 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/plugins/launchpad/test_register.py ... File "/usr/lib/python2.3/site-packages/bzrlib/plugins/launchpad/test_register.py", line 25 from lp_registration import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/repofmt/knitrepo.py ... File "/usr/lib/python2.3/site-packages/bzrlib/repofmt/knitrepo.py", line 17 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/repofmt/weaverepo.py ... File "/usr/lib/python2.3/site-packages/bzrlib/repofmt/weaverepo.py", line 22 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/smart/branch.py ... File "/usr/lib/python2.3/site-packages/bzrlib/smart/branch.py", line 22 from bzrlib.smart.request import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/smart/bzrdir.py ... File "/usr/lib/python2.3/site-packages/bzrlib/smart/bzrdir.py", line 22 from bzrlib.smart.request import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/smart/medium.py ... File "/usr/lib/python2.3/site-packages/bzrlib/smart/medium.py", line 32 from bzrlib.smart.protocol import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/smart/repository.py ... File "/usr/lib/python2.3/site-packages/bzrlib/smart/repository.py", line 25 from bzrlib.smart.request import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/smart/request.py ... File "/usr/lib/python2.3/site-packages/bzrlib/smart/request.py", line 22 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/smart/server.py ... File "/usr/lib/python2.3/site-packages/bzrlib/smart/server.py", line 24 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/smart/vfs.py ... File "/usr/lib/python2.3/site-packages/bzrlib/smart/vfs.py", line 163 backing_bytes = ''.join(bytes for offset, bytes in ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/store/revision/__init__.py ... File "/usr/lib/python2.3/site-packages/bzrlib/store/revision/__init__.py", line 69 @staticmethod ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/store/revision/text.py ... File "/usr/lib/python2.3/site-packages/bzrlib/store/revision/text.py", line 27 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/store/versioned/__init__.py ... File "/usr/lib/python2.3/site-packages/bzrlib/store/versioned/__init__.py", line 27 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/store/__init__.py ... File "/usr/lib/python2.3/site-packages/bzrlib/store/__init__.py", line 33 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/blackbox/__init__.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/blackbox/__init__.py", line 27 from bzrlib.tests import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/blackbox/test_aliases.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/blackbox/test_aliases.py", line 24 from bzrlib.config import (ensure_config_dir_exists, config_filename) ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/blackbox/test_bound_branches.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/blackbox/test_bound_branches.py", line 23 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/blackbox/test_break_lock.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/blackbox/test_break_lock.py", line 22 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/blackbox/test_commit.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/blackbox/test_commit.py", line 24 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/blackbox/test_conflicts.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/blackbox/test_conflicts.py", line 19 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/blackbox/test_export.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/blackbox/test_export.py", line 27 from bzrlib.export import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/blackbox/test_ignore.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/blackbox/test_ignore.py", line 25 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/blackbox/test_info.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/blackbox/test_info.py", line 23 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/blackbox/test_init.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/blackbox/test_init.py", line 23 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/blackbox/test_inventory.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/blackbox/test_inventory.py", line 54 for path, file_id in ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/blackbox/test_logformats.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/blackbox/test_logformats.py", line 23 from bzrlib.config import (ensure_config_dir_exists, config_filename) ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/blackbox/test_merge_directive.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/blackbox/test_merge_directive.py", line 20 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/blackbox/test_mv.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/blackbox/test_mv.py", line 21 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/blackbox/test_push.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/blackbox/test_push.py", line 22 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/blackbox/test_selftest.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/blackbox/test_selftest.py", line 25 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/blackbox/test_serve.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/blackbox/test_serve.py", line 26 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/blackbox/test_tags.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/blackbox/test_tags.py", line 20 from bzrlib.branch import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/blackbox/test_testament.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/blackbox/test_testament.py", line 20 from bzrlib.tests.test_testament import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/blackbox/test_too_much.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/blackbox/test_too_much.py", line 43 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/blackbox/test_upgrade.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/blackbox/test_upgrade.py", line 23 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/branch_implementations/__init__.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/branch_implementations/__init__.py", line 27 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/branch_implementations/test_bound_sftp.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/branch_implementations/test_bound_sftp.py", line 23 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/branch_implementations/test_branch.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/branch_implementations/test_branch.py", line 22 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/branch_implementations/test_create_checkout.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/branch_implementations/test_create_checkout.py", line 19 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/branch_implementations/test_get_revision_id_to_revno_map.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/branch_implementations/test_get_revision_id_to_revno_map.py", line 19 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/bzrdir_implementations/__init__.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/bzrdir_implementations/__init__.py", line 28 from bzrlib.tests import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/bzrdir_implementations/test_bzrdir.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/bzrdir_implementations/test_bzrdir.py", line 26 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/interrepository_implementations/__init__.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/interrepository_implementations/__init__.py", line 28 from bzrlib.repository import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/interrepository_implementations/test_interrepository.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/interrepository_implementations/test_interrepository.py", line 25 from bzrlib.errors import (FileExists, ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/intertree_implementations/__init__.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/intertree_implementations/__init__.py", line 27 from bzrlib.tests import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/intertree_implementations/test_compare.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/intertree_implementations/test_compare.py", line 1017 expected = sorted(self.added(tree2, f_id) for f_id in path_ids) ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/interversionedfile_implementations/__init__.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/interversionedfile_implementations/__init__.py", line 29 from bzrlib.tests import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/per_lock/__init__.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/per_lock/__init__.py", line 24 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/per_lock/test_lock.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/per_lock/test_lock.py", line 19 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/per_lock/test_temporary_write_lock.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/per_lock/test_temporary_write_lock.py", line 19 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/repository_implementations/__init__.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/repository_implementations/__init__.py", line 27 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/repository_implementations/test_iter_reverse_revision_history.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/repository_implementations/test_iter_reverse_revision_history.py", line 19 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/repository_implementations/test_repository.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/repository_implementations/test_repository.py", line 22 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/repository_implementations/test_revision.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/repository_implementations/test_revision.py", line 19 from bzrlib.tests.repository_implementations.test_repository import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/revisionstore_implementations/__init__.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/revisionstore_implementations/__init__.py", line 29 from bzrlib.tests import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/tree_implementations/__init__.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/tree_implementations/__init__.py", line 28 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/tree_implementations/test_get_symlink_target.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/tree_implementations/test_get_symlink_target.py", line 21 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/tree_implementations/test_inv.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/tree_implementations/test_inv.py", line 25 from bzrlib.osutils import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/tree_implementations/test_tree.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/tree_implementations/test_tree.py", line 17 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/workingtree_implementations/__init__.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/workingtree_implementations/__init__.py", line 27 from bzrlib.tests import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/workingtree_implementations/test_add.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/workingtree_implementations/test_add.py", line 21 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/workingtree_implementations/test_basis_tree.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/workingtree_implementations/test_basis_tree.py", line 49 sorted(info[0] for info in basis_tree.list_files(True))) ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/workingtree_implementations/test_commit.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/workingtree_implementations/test_commit.py", line 22 from bzrlib.errors import (NotBranchError, NotVersionedError, ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/workingtree_implementations/test_inv.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/workingtree_implementations/test_inv.py", line 23 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/workingtree_implementations/test_locking.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/workingtree_implementations/test_locking.py", line 19 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/workingtree_implementations/test_move.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/workingtree_implementations/test_move.py", line 21 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/workingtree_implementations/test_readonly.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/workingtree_implementations/test_readonly.py", line 23 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/workingtree_implementations/test_rename_one.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/workingtree_implementations/test_rename_one.py", line 21 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/workingtree_implementations/test_smart_add.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/workingtree_implementations/test_smart_add.py", line 19 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/workingtree_implementations/test_workingtree.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/workingtree_implementations/test_workingtree.py", line 25 from bzrlib.errors import (NotBranchError, NotVersionedError, ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/HTTPTestUtil.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/HTTPTestUtil.py", line 30 from bzrlib.tests.HttpServer import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/__init__.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/__init__.py", line 47 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_annotate.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_annotate.py", line 21 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_atomicfile.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_atomicfile.py", line 23 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_bad_files.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_bad_files.py", line 23 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_branch.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_branch.py", line 27 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_bugtracker.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_bugtracker.py", line 28 @classmethod ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_bundle.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_bundle.py", line 22 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_bzrdir.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_bzrdir.py", line 25 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_cache_utf8.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_cache_utf8.py", line 19 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_commands.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_commands.py", line 20 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_commit.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_commit.py", line 21 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_commit_merge.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_commit_merge.py", line 21 from bzrlib.tests import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_config.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_config.py", line 26 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_conflicts.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_conflicts.py", line 23 from bzrlib.conflicts import (MissingParent, ContentsConflict, TextConflict, ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_decorators.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_decorators.py", line 45 @needs_read_lock ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_delta.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_delta.py", line 20 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_diff.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_diff.py", line 27 from bzrlib.tests import (TestCase, TestCaseWithTransport, ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_dirstate.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_dirstate.py", line 23 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_errors.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_errors.py", line 21 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_extract.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_extract.py", line 17 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_fetch.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_fetch.py", line 21 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_generate_ids.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_generate_ids.py", line 21 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_globbing.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_globbing.py", line 18 from bzrlib.globbing import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_help.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_help.py", line 21 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_http.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_http.py", line 31 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_https_ca_bundle.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_https_ca_bundle.py", line 24 from bzrlib.tests import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_inv.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_inv.py", line 18 from bzrlib.inventory import (Inventory, ROOT_ID, InventoryFile, ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_knit.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_knit.py", line 24 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_lazy_import.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_lazy_import.py", line 22 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_lazy_regex.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_lazy_regex.py", line 21 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_lockable_files.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_lockable_files.py", line 28 from bzrlib.transactions import (PassThroughTransaction, ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_lockdir.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_lockdir.py", line 25 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_log.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_log.py", line 22 from bzrlib.log import (show_log, ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_merge.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_merge.py", line 20 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_merge_core.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_merge_core.py", line 22 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_merge_directive.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_merge_directive.py", line 18 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_options.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_options.py", line 17 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_osutils.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_osutils.py", line 26 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_osutils_encodings.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_osutils_encodings.py", line 25 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_patches.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_patches.py", line 23 from bzrlib.patches import (MalformedLine, ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_progress.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_progress.py", line 21 from bzrlib.progress import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_registry.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_registry.py", line 22 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_remote.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_remote.py", line 28 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_repository.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_repository.py", line 32 from bzrlib.errors import (NotBranchError, ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_revision.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_revision.py", line 21 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_revisionnamespaces.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_revisionnamespaces.py", line 21 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_revisiontree.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_revisiontree.py", line 20 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_rio.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_rio.py", line 31 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_selftest.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_selftest.py", line 28 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_smart_add.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_smart_add.py", line 22 from bzrlib.add import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_smart_transport.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_smart_transport.py", line 26 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_source.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_source.py", line 29 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_ssh_transport.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_ssh_transport.py", line 19 from bzrlib.transport.ssh import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_symbol_versioning.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_symbol_versioning.py", line 24 @symbol_versioning.deprecated_function(symbol_versioning.zero_seven) ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_tag.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_tag.py", line 18 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_timestamp.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_timestamp.py", line 17 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_trace.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_trace.py", line 26 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_transform.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_transform.py", line 21 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_transport.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_transport.py", line 24 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_transport_implementations.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_transport_implementations.py", line 29 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_tree.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_tree.py", line 73 @classmethod ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_ui.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_ui.py", line 27 from bzrlib.progress import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_upgrade.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_upgrade.py", line 29 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_versionedfile.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_versionedfile.py", line 27 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_weave.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_weave.py", line 28 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_whitebox.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_whitebox.py", line 20 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_workingtree.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_workingtree.py", line 21 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_workingtree_4.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_workingtree_4.py", line 22 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tests/test_xml.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tests/test_xml.py", line 19 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/transport/http/__init__.py ... File "/usr/lib/python2.3/site-packages/bzrlib/transport/http/__init__.py", line 32 from bzrlib.transport import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/transport/http/_pycurl.py ... File "/usr/lib/python2.3/site-packages/bzrlib/transport/http/_pycurl.py", line 38 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/transport/http/_urllib.py ... File "/usr/lib/python2.3/site-packages/bzrlib/transport/http/_urllib.py", line 27 from bzrlib.transport.http._urllib2_wrappers import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/transport/http/_urllib2_wrappers.py ... File "/usr/lib/python2.3/site-packages/bzrlib/transport/http/_urllib2_wrappers.py", line 62 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/transport/http/response.py ... File "/usr/lib/python2.3/site-packages/bzrlib/transport/http/response.py", line 149 @staticmethod ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/transport/__init__.py ... File "/usr/lib/python2.3/site-packages/bzrlib/transport/__init__.py", line 53 from bzrlib.symbol_versioning import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/transport/chroot.py ... File "/usr/lib/python2.3/site-packages/bzrlib/transport/chroot.py", line 23 from bzrlib.transport import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/transport/decorator.py ... File "/usr/lib/python2.3/site-packages/bzrlib/transport/decorator.py", line 82 @classmethod ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/transport/fakenfs.py ... File "/usr/lib/python2.3/site-packages/bzrlib/transport/fakenfs.py", line 34 @classmethod ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/transport/fakevfat.py ... File "/usr/lib/python2.3/site-packages/bzrlib/transport/fakevfat.py", line 67 @classmethod ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/transport/ftp.py ... File "/usr/lib/python2.3/site-packages/bzrlib/transport/ftp.py", line 42 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/transport/memory.py ... File "/usr/lib/python2.3/site-packages/bzrlib/transport/memory.py", line 32 from bzrlib.transport import (Transport, register_transport, Server) ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/transport/readonly.py ... File "/usr/lib/python2.3/site-packages/bzrlib/transport/readonly.py", line 37 @classmethod ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/transport/remote.py ... File "/usr/lib/python2.3/site-packages/bzrlib/transport/remote.py", line 29 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/transport/sftp.py ... File "/usr/lib/python2.3/site-packages/bzrlib/transport/sftp.py", line 39 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/transport/ssh.py ... File "/usr/lib/python2.3/site-packages/bzrlib/transport/ssh.py", line 28 from bzrlib.errors import (ConnectionError, ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/ui/__init__.py ... File "/usr/lib/python2.3/site-packages/bzrlib/ui/__init__.py", line 43 from bzrlib.symbol_versioning import (deprecated_method, zero_eight) ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/ui/text.py ... File "/usr/lib/python2.3/site-packages/bzrlib/ui/text.py", line 34 from bzrlib.symbol_versioning import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/version_info_formats/format_python.py ... File "/usr/lib/python2.3/site-packages/bzrlib/version_info_formats/format_python.py", line 21 from bzrlib.version_info_formats import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/version_info_formats/format_rio.py ... File "/usr/lib/python2.3/site-packages/bzrlib/version_info_formats/format_rio.py", line 21 from bzrlib.version_info_formats import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/__init__.py ... File "/usr/lib/python2.3/site-packages/bzrlib/__init__.py", line 46 from bzrlib.symbol_versioning import (deprecated_function, ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/annotate.py ... File "/usr/lib/python2.3/site-packages/bzrlib/annotate.py", line 31 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/branch.py ... File "/usr/lib/python2.3/site-packages/bzrlib/branch.py", line 51 from bzrlib.errors import (BzrError, BzrCheckError, DivergedBranches, ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/builtins.py ... File "/usr/lib/python2.3/site-packages/bzrlib/builtins.py", line 105 @symbol_versioning.deprecated_function(symbol_versioning.zero_fifteen) ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/bzrdir.py ... File "/usr/lib/python2.3/site-packages/bzrlib/bzrdir.py", line 72 from bzrlib.trace import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/commands.py ... File "/usr/lib/python2.3/site-packages/bzrlib/commands.py", line 50 from bzrlib.symbol_versioning import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/commit.py ... File "/usr/lib/python2.3/site-packages/bzrlib/commit.py", line 59 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/conflicts.py ... File "/usr/lib/python2.3/site-packages/bzrlib/conflicts.py", line 201 @staticmethod ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/delta.py ... File "/usr/lib/python2.3/site-packages/bzrlib/delta.py", line 17 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/diff.py ... File "/usr/lib/python2.3/site-packages/bzrlib/diff.py", line 40 from bzrlib.symbol_versioning import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/dirstate.py ... File "/usr/lib/python2.3/site-packages/bzrlib/dirstate.py", line 213 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/errors.py ... File "/usr/lib/python2.3/site-packages/bzrlib/errors.py", line 21 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/externalcommand.py ... File "/usr/lib/python2.3/site-packages/bzrlib/externalcommand.py", line 32 @classmethod ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/fetch.py ... File "/usr/lib/python2.3/site-packages/bzrlib/fetch.py", line 36 from bzrlib.errors import (InstallFailed, ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/generate_ids.py ... File "/usr/lib/python2.3/site-packages/bzrlib/generate_ids.py", line 31 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/globbing.py ... File "/usr/lib/python2.3/site-packages/bzrlib/globbing.py", line 25 from bzrlib.trace import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/help.py ... File "/usr/lib/python2.3/site-packages/bzrlib/help.py", line 27 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/help_topics.py ... File "/usr/lib/python2.3/site-packages/bzrlib/help_topics.py", line 74 lmax = max(len(topic) for topic in topics) ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/identitymap.py ... File "/usr/lib/python2.3/site-packages/bzrlib/identitymap.py", line 21 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/ignores.py ... File "/usr/lib/python2.3/site-packages/bzrlib/ignores.py", line 21 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/info.py ... File "/usr/lib/python2.3/site-packages/bzrlib/info.py", line 22 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/inter.py ... File "/usr/lib/python2.3/site-packages/bzrlib/inter.py", line 70 @classmethod ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/inventory.py ... File "/usr/lib/python2.3/site-packages/bzrlib/inventory.py", line 49 from bzrlib.errors import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/knit.py ... File "/usr/lib/python2.3/site-packages/bzrlib/knit.py", line 73 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/lock.py ... File "/usr/lib/python2.3/site-packages/bzrlib/lock.py", line 40 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/lockable_files.py ... File "/usr/lib/python2.3/site-packages/bzrlib/lockable_files.py", line 22 from bzrlib.decorators import (needs_read_lock, ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/lockdir.py ... File "/usr/lib/python2.3/site-packages/bzrlib/lockdir.py", line 101 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/log.py ... File "/usr/lib/python2.3/site-packages/bzrlib/log.py", line 57 from bzrlib import( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/memorytree.py ... File "/usr/lib/python2.3/site-packages/bzrlib/memorytree.py", line 48 @needs_tree_write_lock ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/merge.py ... File "/usr/lib/python2.3/site-packages/bzrlib/merge.py", line 22 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/merge3.py ... File "/usr/lib/python2.3/site-packages/bzrlib/merge3.py", line 322 @staticmethod ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/merge_directive.py ... File "/usr/lib/python2.3/site-packages/bzrlib/merge_directive.py", line 21 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/mutabletree.py ... File "/usr/lib/python2.3/site-packages/bzrlib/mutabletree.py", line 23 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/option.py ... File "/usr/lib/python2.3/site-packages/bzrlib/option.py", line 285 @staticmethod ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/osutils.py ... File "/usr/lib/python2.3/site-packages/bzrlib/osutils.py", line 21 from stat import (S_ISREG, S_ISDIR, S_ISLNK, ST_MODE, ST_SIZE, ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/remote.py ... File "/usr/lib/python2.3/site-packages/bzrlib/remote.py", line 477 @needs_write_lock ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/repository.py ... File "/usr/lib/python2.3/site-packages/bzrlib/repository.py", line 53 from bzrlib.symbol_versioning import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/revision.py ... File "/usr/lib/python2.3/site-packages/bzrlib/revision.py", line 25 from bzrlib.symbol_versioning import (deprecated_function, ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/revisionspec.py ... File "/usr/lib/python2.3/site-packages/bzrlib/revisionspec.py", line 22 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/revisiontree.py ... File "/usr/lib/python2.3/site-packages/bzrlib/revisiontree.py", line 21 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/rio.py ... File "/usr/lib/python2.3/site-packages/bzrlib/rio.py", line 260 unicode_iter = (line.decode('utf-8') for line in line_iter) ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/status.py ... File "/usr/lib/python2.3/site-packages/bzrlib/status.py", line 19 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tag.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tag.py", line 31 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/testament.py ... File "/usr/lib/python2.3/site-packages/bzrlib/testament.py", line 92 @classmethod ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/textmerge.py ... File "/usr/lib/python2.3/site-packages/bzrlib/textmerge.py", line 100 @staticmethod ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/trace.py ... File "/usr/lib/python2.3/site-packages/bzrlib/trace.py", line 64 from bzrlib.symbol_versioning import (deprecated_function, ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/transform.py ... File "/usr/lib/python2.3/site-packages/bzrlib/transform.py", line 30 from bzrlib.errors import (DuplicateKey, MalformedTransform, NoSuchFile, ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tree.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tree.py", line 25 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/tsort.py ... File "/usr/lib/python2.3/site-packages/bzrlib/tsort.py", line 367 self._revnos = dict((revision, [None, 0]) for revision in self._graph) ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/urlutils.py ... File "/usr/lib/python2.3/site-packages/bzrlib/urlutils.py", line 535 _hex_display_map.update((hex,'%'+hex) for hex in _no_decode_hex) ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/version.py ... File "/usr/lib/python2.3/site-packages/bzrlib/version.py", line 23 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/versionedfile.py ... File "/usr/lib/python2.3/site-packages/bzrlib/versionedfile.py", line 39 from bzrlib.symbol_versioning import (deprecated_function, ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/weave.py ... File "/usr/lib/python2.3/site-packages/bzrlib/weave.py", line 78 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/workingtree.py ... File "/usr/lib/python2.3/site-packages/bzrlib/workingtree.py", line 93 from bzrlib.osutils import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/workingtree_4.py ... File "/usr/lib/python2.3/site-packages/bzrlib/workingtree_4.py", line 74 from bzrlib.osutils import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/xml5.py ... File "/usr/lib/python2.3/site-packages/bzrlib/xml5.py", line 20 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/xml7.py ... File "/usr/lib/python2.3/site-packages/bzrlib/xml7.py", line 17 from bzrlib import ( ^ SyntaxError: invalid syntax Compiling /usr/lib/python2.3/site-packages/bzrlib/xml_serializer.py ... File "/usr/lib/python2.3/site-packages/bzrlib/xml_serializer.py", line 30 from xml.etree.cElementTree import (ElementTree, SubElement, Element, ^ SyntaxError: invalid syntax INFO: using old version '/usr/bin/python2.3' INFO: using unsupported version '/usr/bin/python2.5' pycentral: pycentral pkginstall: error byte-compiling files (536) pycentral pkginstall: error byte-compiling files (536) dpkg: error processing python-bzr (--install): subprocess post-installation script returned error exit status 1 Errors were encountered while processing: python-bzr ii bicyclerepair 0.9-4.1 ii idle-python2.3 2.3.5-15 ii idle-python2.4 2.4.4-3 ii ipython 0.7.3-2 ii libboost-python1.33.1 1.33.1-10 ii pylint 0.12.1-1 ii python 2.4.4-2 ii python-4suite-common 0.99cvs20060405-1.1 ii python-4suite-doc 1.0~rc4cvs20061016-1 ii python-all 2.4.4-2 ii python-all-dev 2.4.4-2 ii python-apt 0.6.20 ii python-cairo 1.2.0-1 ii python-central 0.5.13-0.1 ii python-chm 0.8.4-0.1 ii python-cjkcodecs 1.1.1-2+b1 ii python-dateutil 1.1-1 ii python-dbus 0.80.2-2 ii python-dev 2.4.4-2 ii python-doc 2.4.4-1 ii python-ecasound2.2 2.4.4-6 ii python-egenix-mxdatetime 2.0.6-4 ii python-egenix-mxtools 2.0.6-4 ii python-gd 0.52debian-3 ii python-glade2 2.8.6-8 ii python-gmenu 2.16.1-3 ii python-gnome2 2.12.4-6 ii python-gnome2-desktop 2.14.0-3 ii python-gnome2-extras 2.14.3-1 ii python-gnuplot 1.7-7 ii python-gtk2 2.8.6-8 ii python-gtk2-tutorial 2.4-1 ii python-gtkmvc 1.0.0-1 ii python-id3 1.2-6.2 ii python-imaging 1.1.5-11 ii python-imaging-tk 1.1.5-11 ii python-kiwi 1.9.13-2 ii python-libxml2 2.6.27.dfsg-1 ii python-logilab-astng 0.16.2-2 ii python-logilab-common 0.21.0-2 ii python-markdown 1.4-2 ii python-matplotlib 0.87.7-0.3 ii python-matplotlib-data 0.87.7-0.3 ii python-matplotlib-doc 0.87.7-0.3 ii python-minimal 2.4.4-2 ii python-mode 1.0-3.1 ii python-newt 0.52.2-10 ii python-numarray 1.5.2-2.2 ii python-numarray-doc 1.5.2-2.2 ii python-numeric 24.2-7 ii python-numeric-ext 24.2-7 ii python-numpy 1.0.1-8 ii python-numpy-dev 1.0.1-8 ii python-numpy-ext 1.0.1-8 ii python-pexpect 2.1-1 ii python-profiler 2.4.4-3 ii python-pycurl 7.15.5-1 ii python-pyorbit 2.0.1-5 ii python-pyrex 0.9.5.1a-1 ii python-pysqlite2 2.3.3-1 ii python-qt3 3.16-1.2 ii python-rrd 1.2.19-1 ii python-setuptools 0.6c5-2 ii python-sip4 4.4.5-4 ii python-stdeb 0.2.a1-1 ii python-support 0.6.3 ii python-tk 2.4.4-1 ii python-tz 2006p-0.1 ii python-uno 2.0.4.dfsg.2-7 ii python-wxgtk2.6 2.6.3.2.1.5 ii python-wxtools 2.6.3.2.1.5 ii python-wxversion 2.6.3.2.1.5 ii python-xml 0.8.4-6 ii python2.3 2.3.5-15 ii python2.3-dev 2.3.5-15 ii python2.4 2.4.4-3 ii python2.4-dev 2.4.4-3 ii python2.4-doc 2.4.4-2 ii python2.4-minimal 2.4.4-3 ii python2.5 2.5-5 ii python2.5-minimal 2.5-5 From maya_c_c at yahoo.com Thu May 10 07:49:29 2007 From: maya_c_c at yahoo.com (maya lan) Date: Wed, 9 May 2007 22:49:29 -0700 (PDT) Subject: [Distutils] stdeb- how to? Message-ID: <817423.1408.qm@web30502.mail.mud.yahoo.com> Dear all... i am a new in this... how i can use stdeb to create a debian package for my python package(my project).. i read this (http://stdeb.python-hosting.com/) article.. but i cant understand.. so plz help thanks in advance. maya ____________________________________________________________________________________ 8:00? 8:25? 8:40? Find a flick in no time with the Yahoo! Search movie showtime shortcut. http://tools.search.yahoo.com/shortcuts/#news From matt at matt-good.net Thu May 10 08:38:39 2007 From: matt at matt-good.net (Matt Good) Date: Wed, 09 May 2007 23:38:39 -0700 Subject: [Distutils] stdeb- how to? In-Reply-To: <817423.1408.qm@web30502.mail.mud.yahoo.com> References: <817423.1408.qm@web30502.mail.mud.yahoo.com> Message-ID: <4642BDEF.3030108@matt-good.net> maya lan wrote: > Dear all... > i am a new in this... > how i can use stdeb to create a debian package for my > python package(my project).. > i read this (http://stdeb.python-hosting.com/) > article.. but i cant understand.. so plz help > How far have you gotten? I haven't tried this package, but from what I read on the page and what I know of Debian packaging I'd suggest running the following: stdeb_run_setup debuild *.dsc If you don't have debuild installed try this instead of debuild: dpkg-source -x *.dsc cd dpkg-buildpackage -rfakeroot This list is good for help directly with distutils or setuptools, and some with zc.buildout, but I'm not sure what you'll find with an extension like stdeb, since I don't think it's well known. You can also try contacting the author directly (his email is on the website). Or if you need help with Debian packaging there is the Debian New Maintainers Guide, or take a look at the Ubuntu Packaging Guide which is not as comprehensive, but a little clearer introduction: http://doc.ubuntu.com/ubuntu/packagingguide/C/index.html -- Matt Good From strawman at astraw.com Fri May 11 01:07:08 2007 From: strawman at astraw.com (Andrew Straw) Date: Thu, 10 May 2007 16:07:08 -0700 Subject: [Distutils] stdeb bug: syntax error during dpkg -i In-Reply-To: <464232C3.7030300@pvv.org> References: <464232C3.7030300@pvv.org> Message-ID: <4643A59C.3010307@astraw.com> Dear Tore, I think the issue you're having is that bzr-0.16 requires, syntax-wise, Python 2.4 or greater, but your pycentral is configured to install things for Python2.3. Specifically, the postinst file generated by dh_pycentral calls each Python interpreter to compile the bytecode of each source file. (I don't know if there's a way to tell pycentral not to use Python2.3, but we can ask how if indeed this is the problem you're facing.) What is the output of "pycentral showversions"? FWIW, I built a version of python-bzr_0.16.0-1_all.deb using more-or-less the commands you issued just now on Ubuntu Edgy. Upon installing it, all went well, and I didn't get the issues you're having. Also, I just added a section "A complete example" to the webpage http://stdeb.python-hosting.com/ using your package as an example. This works fine on my system. Tore Ferner wrote: > Hi, > > stdeb seems like a good idea... > > I tried to use 0.2.a1 on bzr > http://bazaar-vcs.org/releases/src/bzr-0.16.tar.gz > and got the error message below. > I run Debian unstable; I've paused upgrading since just after Etch was > released. At the end you can see my installed python modules. > On my system python equals python2.4, but for some things python2.3 is > used (for some reason unknown to me, but it has worked fine so far). > > Generated thus: > > python -c "import stdeb; execfile('setup.py')" sdist_dsc > cd deb_dist/bzr-0.16.0/ > dpkg-buildpackage -tc > dpkg -i ../python-bzr_0.16.0-1_all.deb > > Best regards > Tore > > > > (Reading database ... 242979 files and directories currently installed.) > Preparing to replace python-bzr 0.16.0-1 (using > python-bzr_0.16.0-1_all.deb) ... > INFO: using old version '/usr/bin/python2.3' > INFO: using unsupported version '/usr/bin/python2.5' > Unpacking replacement python-bzr ... > Setting up python-bzr (0.16.0-1) ... > Compiling > /usr/lib/python2.3/site-packages/bzrlib/benchmarks/tree_creator/__init__.py > ... > File > "/usr/lib/python2.3/site-packages/bzrlib/benchmarks/tree_creator/__init__.py", > line 25 > from bzrlib import ( > ^ > SyntaxError: invalid syntax > From strawman at astraw.com Fri May 11 02:11:44 2007 From: strawman at astraw.com (Andrew Straw) Date: Thu, 10 May 2007 17:11:44 -0700 Subject: [Distutils] stdeb- how to? In-Reply-To: <817423.1408.qm@web30502.mail.mud.yahoo.com> References: <817423.1408.qm@web30502.mail.mud.yahoo.com> Message-ID: <4643B4C0.1000001@astraw.com> Dear Maya, A just added a section "A complete example" to the webpage http://stdeb.python-hosting.com/ . If you have further questions after reading that example, please ask! -Andrew maya lan wrote: > Dear all... > i am a new in this... > how i can use stdeb to create a debian package for my > python package(my project).. > i read this (http://stdeb.python-hosting.com/) > article.. but i cant understand.. so plz help > > > thanks in advance. > maya > > > > > ____________________________________________________________________________________ > 8:00? 8:25? 8:40? Find a flick in no time > with the Yahoo! Search movie showtime shortcut. > http://tools.search.yahoo.com/shortcuts/#news > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig From ct at gocept.com Fri May 11 10:21:42 2007 From: ct at gocept.com (Christian Theune) Date: Fri, 11 May 2007 10:21:42 +0200 Subject: [Distutils] setuptools process_url doesn't handle URLErrors gracefullly Message-ID: <1178871702.22778.8.camel@mindy.whq.gocept.com> Hi, I'm fighting against a problem where the DNS name of the homepage of a package that is listed on the cheeseshop isn't accessible any more. In this special case the DNS name for this thing is gone (the package is `rwproperty` the missing DNS entry is z3lab.org). In this case easy_install dies with the following error, although the eggs are listed on the cheeseshop page and the listed homepage is pretty irrelevant: Searching for rwproperty Reading http://cheeseshop.python.org/pypi/rwproperty/ Reading http://www.z3lab.org/sections/blogs/philipp-weitershausen/2006_05_29_pycon-06-lightning-talk Traceback (most recent call last): File "/usr/bin/easy_install", line 7, in ? sys.exit( File "/usr/lib/python2.4/site-packages/setuptools-0.6c5-py2.4.egg/setuptools/command/easy_install.py", line 1670, in main with_ei_usage(lambda: File "/usr/lib/python2.4/site-packages/setuptools-0.6c5-py2.4.egg/setuptools/command/easy_install.py", line 1659, in with_ei_usage return f() File "/usr/lib/python2.4/site-packages/setuptools-0.6c5-py2.4.egg/setuptools/command/easy_install.py", line 1674, in distclass=DistributionWithoutHelpCommands, **kw File "/usr/lib/python2.4/distutils/core.py", line 149, in setup dist.run_commands() File "/usr/lib/python2.4/distutils/dist.py", line 946, in run_commands self.run_command(cmd) File "/usr/lib/python2.4/distutils/dist.py", line 966, in run_command cmd_obj.run() File "/usr/lib/python2.4/site-packages/setuptools-0.6c5-py2.4.egg/setuptools/command/easy_install.py", line 211, in run self.easy_install(spec, not self.no_deps) File "/usr/lib/python2.4/site-packages/setuptools-0.6c5-py2.4.egg/setuptools/command/easy_install.py", line 432, in easy_install dist = self.package_index.fetch_distribution( File "/usr/lib/python2.4/site-packages/setuptools-0.6c5-py2.4.egg/setuptools/package_index.py", line 462, in fetch_distribution self.find_packages(requirement) File "/usr/lib/python2.4/site-packages/setuptools-0.6c5-py2.4.egg/setuptools/package_index.py", line 303, in find_packages self.scan_url(self.index_url + requirement.unsafe_name+'/') File "/usr/lib/python2.4/site-packages/setuptools-0.6c5-py2.4.egg/setuptools/package_index.py", line 610, in scan_url self.process_url(url, True) File "/usr/lib/python2.4/site-packages/setuptools-0.6c5-py2.4.egg/setuptools/package_index.py", line 201, in process_url page = self.process_index(url, page) File "/usr/lib/python2.4/site-packages/setuptools-0.6c5-py2.4.egg/setuptools/package_index.py", line 278, in process_index self.scan_url(new_url) File "/usr/lib/python2.4/site-packages/setuptools-0.6c5-py2.4.egg/setuptools/package_index.py", line 610, in scan_url self.process_url(url, True) File "/usr/lib/python2.4/site-packages/setuptools-0.6c5-py2.4.egg/setuptools/package_index.py", line 191, in process_url self.fetched_urls[url] = self.fetched_urls[f.url] = True AttributeError: URLError instance has no attribute 'url' I didn't find any switch to turn to make this go away and I don't know the setuptools code so my few feeble attempts to fix this didn't work out. Some help would be very appreciated. Christian -- gocept gmbh & co. kg - forsterstra?e 29 - 06112 halle/saale - germany www.gocept.com - ct at gocept.com - phone +49 345 122 9889 7 - fax +49 345 122 9889 1 - zope and plone consulting and development -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Dies ist ein digital signierter Nachrichtenteil Url : http://mail.python.org/pipermail/distutils-sig/attachments/20070511/7ea3207a/attachment.pgp From strawman at astraw.com Fri May 11 12:05:19 2007 From: strawman at astraw.com (Andrew Straw) Date: Fri, 11 May 2007 03:05:19 -0700 Subject: [Distutils] stdeb bug: syntax error during dpkg -i In-Reply-To: <4643A59C.3010307@astraw.com> References: <464232C3.7030300@pvv.org> <4643A59C.3010307@astraw.com> Message-ID: <46443FDF.2010104@astraw.com> Tore Ferner wrote: > Andrew Straw wrote: > >> Dear Tore, >> >> I think the issue you're having is that bzr-0.16 requires, syntax-wise, >> Python 2.4 or greater, but your pycentral is configured to install >> things for Python2.3. Specifically, the postinst file generated by >> dh_pycentral calls each Python interpreter to compile the bytecode of >> each source file. (I don't know if there's a way to tell pycentral not >> to use Python2.3, but we can ask how if indeed this is the problem >> you're facing.) What is the output of "pycentral showversions"? >> > > Thank you for your response! > > $ pycentral showversions > 2.4 > pycentral showversions is deprecated, use `pyversions -vs' > > $ pyversions -vs > 2.4 > > OK, I'll read up a little on pycentral and see if I need to fix my system... > Do you have python2.3 installed on your system? If you have, then there > probably is something odd with my machine. > Hmm, this is strange. Apparently pycentral isn't interested in Python2.3, and yet it does get called. It's possible there's a pycentral bug, but I'm not sure. Can you post the contents of your deb-dst/bzr-0.16.0/debian/rules file? I don't have Python2.3 installed, so I'm flying blindly here... > >> FWIW, I built a version of python-bzr_0.16.0-1_all.deb using >> more-or-less the commands you issued just now on Ubuntu Edgy. Upon >> installing it, all went well, and I didn't get the issues you're having. >> >> Also, I just added a section "A complete example" to the webpage >> http://stdeb.python-hosting.com/ using your package as an example. This >> works fine on my system. >> > > Yes, that's helpful. Thank you! > > I have a couple of questions; if you have time, maybe you could add > information about this to your on-line example?: > > 1) bzr is, I suppose, a standalone program rather than a python module. > It would be proper to name the package "bzr", "bzr-local", "bzr-mybuild" > or something. I think "bzr" would be best, or perhaps that it at least > provides bzr. > What would be the best way to achieve this with stdeb? > Hmm, I'll have to add this. This shouldn't hard. > 2) Would it be possible to get the source documentation under doc/ > placed in /usr/share/doc/package-name (e.g. /usr/share/doc/python-bzr)? > Again, this is another feature request -- I'll have to add support for it. Unfortunately, there's not a universal standard for Python documentation (separate from the source code), so this one will require some thinking about. > 3) How to shift the installation from /usr to /usr/local? (? la > --prefix=/usr/local) > Hmm, another good question... Does pycentral actually support that? I guess it does. OK, I'll consider this another feature request! :) I'll send an email when I've implemented any or all of these features... -Andrew From jim at zope.com Sat May 12 16:34:00 2007 From: jim at zope.com (Jim Fulton) Date: Sat, 12 May 2007 10:34:00 -0400 Subject: [Distutils] Overly picky about file URLs Message-ID: <80498EC4-E2E5-42E2-BA3F-A25CBB7B4200@zope.com> The local_open function in setuptools.package_index is used to open file URLs. A file URL pointing to a directory is treated as non found unless it ends in a "/". This seems overly restrictive to me. Would anyone object if the restriction was relaxed? Jim -- Jim Fulton mailto:jim at zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From peter.mowry at amd.com Sun May 13 02:09:55 2007 From: peter.mowry at amd.com (Mowry, Peter) Date: Sat, 12 May 2007 19:09:55 -0500 Subject: [Distutils] Python module to: Talk to stdout/stdin for 64-bit Windows/Linux C++ application? Message-ID: <475F4CA05E4E1B46A7213674E70EFA62022968CA@SAUSEXMB2.amd.com> Hi, Background / Goal: My team's application does regressions by controlling our 64-bit Windows/Linux C++ application (SimNow, www.amd.com/simnow) via stdout/stdin with ("pexpect" for unix) and (python COM objects) for Windows (b/c popen doesn't work). However, I am trying to replace this with a unified windows/linux solution. The idea is to extend Python with a SimNow-control module. My sole goal in extending a python module is so our python regressions scripts can use python to control SimNow via SimNow's stdin/stdout interface. Sample Final results: import simnow simnow.start() # start simnow executable output1 = simnow.sendCommand("load platform x") # blocking call, sends simnow cmd, returns the output string Disutils: Does extending python with Disutils sound like the simple and clean way for my goal? Is there a simple cookbook solution for my goal? (Btw, someone on my team already embedded python into SimNow. But I think we actually want to extend python, so to control SimNow from our python regressions code/infrastructure.) --Thanks, any response is appreciated! Peter Mowry ("Pem") SimNow Team, Software Engineer AMD Organization -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20070512/3f950f6a/attachment.htm -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/gif Size: 73 bytes Desc: image002.gif Url : http://mail.python.org/pipermail/distutils-sig/attachments/20070512/3f950f6a/attachment.gif From david at mantara.com Sun May 13 04:59:39 2007 From: david at mantara.com (David Arnold) Date: Sun, 13 May 2007 12:59:39 +1000 Subject: [Distutils] Python module to: Talk to stdout/stdin for 64-bit Windows/Linux C++ application? In-Reply-To: Your message of "Sat, 12 May 2007 19:09:55 EST." <475F4CA05E4E1B46A7213674E70EFA62022968CA@SAUSEXMB2.amd.com> Message-ID: <15086.1179025179@d.0x1.org> -->"Peter" == Mowry, Peter writes: Peter> My team's application does regressions by controlling our Peter> 64-bit Windows/Linux C++ application (SimNow, Peter> www.amd.com/simnow) via stdout/stdin with ("pexpect" for unix) Peter> and (python COM objects) for Windows (b/c popen doesn't work).? Peter> However, I am trying to replace this with a unified Peter> windows/linux solution. have you looked at the subprocess module, part of the standard library in Python 2.4 and later? Peter> Does extending python with Disutils sound like the simple and Peter> clean way for my goal?? Is there a simple cookbook solution for Peter> my goal? i don't think distutils is what you're after. distutils is about producing installers for Python applications. d From cito at online.de Sun May 13 07:44:34 2007 From: cito at online.de (Christoph Zwerschke) Date: Sun, 13 May 2007 07:44:34 +0200 Subject: [Distutils] bdist_eggs soiling pure packages In-Reply-To: <20070509182418.3CF133A4061@sparrow.telecommunity.com> References: <463C590D.7030408@online.de> <20070509182418.3CF133A4061@sparrow.telecommunity.com> Message-ID: <4646A5C2.30300@online.de> Phillip J. Eby wrote: > Christoph Zwerschke wrote: >> ... >> I think the real problem is that setuptools replaces the shebang line >> when *building* the egg, not when *installing* the egg. > > Use entry point scripts instead; they don't have this problem: > > http://peak.telecommunity.com/DevCenter/setuptools#automatic-script-creation Ok, and they have some other advantages as well, so I just suggested to change this in TurboGears (http://trac.turbogears.org/ticket/1386). Anyway, I think this is a flaw in distutils that should be fixed if it's not too difficult. -- Chris From a.schmolck at gmx.net Sun May 13 17:01:07 2007 From: a.schmolck at gmx.net (Alexander Schmolck) Date: Sun, 13 May 2007 16:01:07 +0100 Subject: [Distutils] unpacking files from and egg only for testing Message-ID: Hi, I'm currently working on a package making the move from distutils to setuptools and I've got the following problem: although the code itself should work fine out of an egg, testing requires some a directory with some files to be created for matlab. Since it's only needed for testing I don't want to clutter the installers HD. How should I best handle this? Currently I'm just using ``zip_safe=False``, but a more elegant solution would seem to be for the test-code to temporarily unpack the relevant files from the egg into some temp-dir run the tests and delete the temp files again. I figure other projects might face a similar situation and I was wondering if some good solution is already available, or if I just should hack something together. cheers 'as From a.schmolck at gmx.net Sun May 13 17:14:09 2007 From: a.schmolck at gmx.net (Alexander Schmolck) Date: Sun, 13 May 2007 16:14:09 +0100 Subject: [Distutils] automating __version__' ing issues Message-ID: Hi, I'm still trying to figure out how to handle versioning, including development release versioning in a simple but automated fashion. Basically assuming svn had some hypothetical, $Version$ tag, which would either list the user-chosen version name, plus, if there have been changes since that has been last assigned, a the project-wide svn-release number (e.g. "1.1" or "1.1.dev43"), I'd just put that in setup.py, the project's __version__ = ... string in __init__.py and ideally even the docs. No such thing exists, however and so far I've been doing it by hand, which is clearly suboptimal, especially since I'm also currently writing a howto for scikits authors ( , very rough at the moment) were I'd like to give best practices. Basically, if svn had a $Revision$ keyword that worked, I'd just use that in setup.py and the package's __init__ (and possibly the README). Now numpy.distutils.utils contains some code to create a __svn_version__.py file which can then be used in conjunction with a version.py file, in which one specifies the official version number + a release flag (true or false) to get either "1.1" or "1.1.dev43". I've started to detail this at , but it seems rather clumsy and also has the disadvantage that one needs to use numpy.distutils in addtition to setuptools, further complicating things (numpy.distutils is also not that well documented IMO, and I haven't got the approach to work as I would want). So I was wondering if there wasn't maybbe a nicer way to achieve this, using just setuptools. I've had a look at but from that it's e.g. not clear to me how one would obtain the right __version__ string in ``project/__init__.py``. I think I might just have missed the right info in the docs, but I'd really appreciate it if someone could walk me through the steps how I can specify at one place that I want to create a proper release with version X or a development release with version X.devY and have this info available everywhere it's needed. cheers, 'as From lxander.m at gmail.com Sun May 13 17:44:29 2007 From: lxander.m at gmail.com (Alexander Michael) Date: Sun, 13 May 2007 11:44:29 -0400 Subject: [Distutils] unpacking files from and egg only for testing In-Reply-To: References: Message-ID: <525f23e80705130844x7331e5c5vf85544a23fc6378a@mail.gmail.com> On 5/13/07, Alexander Schmolck wrote: > Hi, > > I'm currently working on a package making the > move from distutils to setuptools and I've got the following problem: although > the code itself should work fine out of an egg, testing requires some a > directory with some files to be created for matlab. Since it's only needed for > testing I don't want to clutter the installers HD. > > How should I best handle this? Currently I'm just using ``zip_safe=False``, > but a more elegant solution would seem to be for the test-code to temporarily > unpack the relevant files from the egg into some temp-dir run the tests and > delete the temp files again. I figure other projects might face a similar > situation and I was wondering if some good solution is already available, or > if I just should hack something together. I don't think setuptools/pkg_resources/eggs are designed to handle running tests like this after (end-user) installation. There is support for running unittests during development as part of package building. Does the end user *need* to run these tests to determine the correct package configuration for their machine? If not, you should probably wrap up the tests into a test suite and use the setuptools test functionality () along with tempfile (), otherwise I don't think (but could have missed something) that eggs support running arbitrary code at install time to do such configuration (hence Enthought's postinstall extensions). If the latter, perhaps someone else can be of more help. Alex From lxander.m at gmail.com Sun May 13 17:45:37 2007 From: lxander.m at gmail.com (Alexander Michael) Date: Sun, 13 May 2007 11:45:37 -0400 Subject: [Distutils] automating __version__' ing issues In-Reply-To: References: Message-ID: <525f23e80705130845j52fdb809td3c9de54abcd4aa5@mail.gmail.com> On 5/13/07, Alexander Schmolck wrote: > Hi, > > I'm still trying to figure out how to handle versioning, including development > release versioning in a simple but automated fashion. Basically assuming svn > had some hypothetical, $Version$ tag, which would either list the user-chosen > version name, plus, if there have been changes since that has been last > assigned, a the project-wide svn-release number (e.g. "1.1" or "1.1.dev43"), > I'd just put that in setup.py, the project's __version__ = ... string in > __init__.py and ideally even the docs. No such thing exists, however and so > far I've been doing it by hand, which is clearly suboptimal, especially since > I'm also currently writing a howto for scikits authors ( > , very rough at the moment) were I'd > like to give best practices. > > Basically, if svn had a $Revision$ keyword that worked, I'd just use that in > setup.py and the package's __init__ (and possibly the README). Now > numpy.distutils.utils contains some code to create a __svn_version__.py file > which can then be used in conjunction with a version.py file, in which one > specifies the official version number + a release flag (true or false) to get > either "1.1" or "1.1.dev43". I've started to detail this at > , but it seems rather > clumsy and also has the disadvantage that one needs to use numpy.distutils in > addtition to setuptools, further complicating things (numpy.distutils is also > not that well documented IMO, and I haven't got the approach to work as I > would want). > > So I was wondering if there wasn't maybbe a nicer way to achieve this, using > just setuptools. I've had a look at > > > > but from that it's e.g. not clear to me how one would obtain the right > __version__ string in ``project/__init__.py``. I think I might just have > missed the right info in the docs, but I'd really appreciate it if someone > could walk me through the steps how I can specify at one place that I want to > create a proper release with version X or a development release with version > X.devY and have this info available everywhere it's needed. SVN does support keywords, but you need to turn-on keyword interpolation (see the SVN documentation describing properties for how to do this), but I don't think you'll need this functionality. Development release suffixes can be added with setuptools (). When using this technique, the setup.py version string must be the *next* version (the one under development that is typically your SVN Trunk). Here's the rub-- it must be in setup.py and you probably want to give your package access to it so that it can be reported. It appears that the current idiom for solving this dilemma is to but a release.py or version.py file in your package, which is sucked up into the setup.py file with execfile (see for an example). You then manually maintain the version number in one place (the release.py file). Also note that it appears the current trend is to keep __init__.py files empty (required if you ever want to create a setuptools namespace package) and create an api.py file with convenient exports. This is done to avoid running into issues with circular references. I would be tempted to but version information my __init__.py file (and only version information), but this will breakdown if the package ever grows to become a namespace package. Perhaps avoiding this simplification, however, is over future-proofing. From lxander.m at gmail.com Sun May 13 17:57:22 2007 From: lxander.m at gmail.com (Alexander Michael) Date: Sun, 13 May 2007 11:57:22 -0400 Subject: [Distutils] unpacking files from and egg only for testing In-Reply-To: References: <525f23e80705130819r2d954394l654f0a4b5d94eca3@mail.gmail.com> Message-ID: <525f23e80705130857pdfb6b4ew226caa8ef2729ae7@mail.gmail.com> [brought back on-list (forgot/didn't notice this list doesn't set Reply-To)] On 5/13/07, Alexander Schmolck wrote: > [did you send this off-list on purpose?] > "Alexander Michael" writes: > > > I don't think setuptools/pkg_resources/eggs are designed to handle > > running tests like this after (end-user) installation. There is > > support for running unittests during development as part of package > > building. Does the end user *need* to run these tests to determine the > > correct package configuration for their machine? > > No. > > > If not, you should probably wrap up the tests into a test suite and use the > > setuptools test functionality > > () > > I do have a unitest test suite, and I'm also using the test_suite option. > However that test suite needs to place several, exiting .m files into tempdir > (and tell matlab about their whereabouts in order to run all the tests, but > telling matlab is easy). I still don't see a good way do conditionally extract > stuff from the egg for testing (I don't think ``test_suite`` does that, unless > I'm missing something). I could obviously encode all these files as a python > string in the testmodule and then create them manually in the tempdir or > something similarly horrible -- sort of like a shell archive, but that strikes > me as suboptimal. We're getting out of my depth here. But it is my understanding that the test command adds the source directory to the sys.path, so that you could acquire undeclared data files the old-fashioned way (i.e. os.path.join(mypackage.__path__, 'test-mex.c'), although you might be able to use pkg_resources which is cleaner. Your test would copy the test data to a tempdir, do its thing, check the result, and clean-up. > > along with tempfile (), > > otherwise I don't think (but could have missed something) that eggs support > > running arbitrary code at install time to do such configuration (hence > > Enthought's postinstall extensions). If the latter, perhaps someone else can > > be of more help. From optilude at gmx.net Sun May 13 18:00:01 2007 From: optilude at gmx.net (Martin Aspeli) Date: Sun, 13 May 2007 17:00:01 +0100 Subject: [Distutils] Reverse dependencies Message-ID: Hi, I'm finding myself in the following situation. I'm writing package my.app, which encapsulates higher level concepts specific to my application. My application runs on Plone, targeting version 3, which runs on Zope 2.10 and sits on top of CMF 2.1. Now, I want to depend on your.package, which provides some autonomous functionality, say support for a particular use case or application policy. I want to be able to get bugfixes and incremental improvements for your.package in the future, and when other people install my.app I'd like them to have the latest appropriate version of your.package. Let's say your.package version 1.5 targets Plone 3 and Zope 2.10. At this point I can't know if there'll be a 1.6 or a 1.7 or whatever also targeting Plone 3. At some point, maybe version 2.0, maybe 1.8, maybe 4.5, it's possible that your.package will stop working on Plone 3. At this point, someone installing my.app may inadvertently get an incompatible version of your.package, if I have a broad (or lazy) install_requires. What I'd like to be able to say, is "I run on Plone 3, CMF 2.1, Zope 2.10, Python 2.5" -- these are my "framework" realities -- and I'd like the latest version of your.package supporting this platform. For that to work, your.package would need to declare such compatibility. If it doesn't, I'd need to research and encode this in my own install_requires. I may need to change it in the future, when the roadmap for your.package becomes more clear. The safe choice is possibly to declare a very narrow version (>=1.5<1.6), at the risk of losing out on bug fixes. But is there any support for such declarative "reverse" dependencies? Is it something worth exploring further? Martin From pje at telecommunity.com Sun May 13 18:23:04 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 13 May 2007 12:23:04 -0400 Subject: [Distutils] unpacking files from and egg only for testing In-Reply-To: References: Message-ID: <20070513162117.C061E3A40B0@sparrow.telecommunity.com> At 04:01 PM 5/13/2007 +0100, Alexander Schmolck wrote: >Hi, > >I'm currently working on a package making the >move from distutils to setuptools and I've got the following problem: although >the code itself should work fine out of an egg, testing requires some a >directory with some files to be created for matlab. Since it's only needed for >testing I don't want to clutter the installers HD. > >How should I best handle this? Currently I'm just using ``zip_safe=False``, >but a more elegant solution would seem to be for the test-code to temporarily >unpack the relevant files from the egg into some temp-dir run the tests and >delete the temp files again. Use pkg_resources.resource_filename() - it will give you a real filename for the resource (unpacked to a temporary directory if necessary). If you are running "setup.py test", on the other hand, it will return the path to the original source file. From pje at telecommunity.com Sun May 13 18:30:21 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 13 May 2007 12:30:21 -0400 Subject: [Distutils] automating __version__' ing issues In-Reply-To: <525f23e80705130845j52fdb809td3c9de54abcd4aa5@mail.gmail.co m> References: <525f23e80705130845j52fdb809td3c9de54abcd4aa5@mail.gmail.com> Message-ID: <20070513162836.041E23A40B0@sparrow.telecommunity.com> At 11:45 AM 5/13/2007 -0400, Alexander Michael wrote: >SVN does support keywords, but you need to turn-on keyword >interpolation (see the SVN documentation describing properties for how >to do this), but I don't think you'll need this functionality. >Development release suffixes can be added with setuptools >(). > When using this technique, the setup.py version string must be the >*next* version (the one under development that is typically your SVN >Trunk). It only has to be the next version if you're using a prerelease suffix like "dev". For example "1.2-r5499" is SVN revision 5499 of an already-released version 1.2. If you *haven't* released 1.2, you want to call it '1.2.dev-r5499' instead, which compares less than '1.2', e.g.: >>> from pkg_resources import parse_version as pv >>> pv('1.2') > pv('1.2dev') True >>> pv('1.2-r5499') > pv('1.2') True >>> pv('1.2-r5499') > pv('1.2dev-r5499') True >Here's the rub-- it must be in setup.py and you probably want >to give your package access to it so that it can be reported. It >appears that the current idiom for solving this dilemma is to but a >release.py or version.py file in your package, which is sucked up into >the setup.py file with execfile (see > for an >example). You then manually maintain the version number in one place >(the release.py file). Yep, this is what most people do. I actually use a PEAK tool called "version" (peak.tools.version) which maintains a data file with the version, and does targeted search-and-replace operations on the files listed in a configuration file. This is a lot easier when you have documentation files that also need version information updated, and it also avoids the execfile pain. From a.schmolck at gmx.net Sun May 13 18:30:03 2007 From: a.schmolck at gmx.net (Alexander Schmolck) Date: Sun, 13 May 2007 17:30:03 +0100 Subject: [Distutils] automating __version__' ing issues In-Reply-To: <525f23e80705130845j52fdb809td3c9de54abcd4aa5@mail.gmail.com> (Alexander Michael's message of "Sun\, 13 May 2007 11\:45\:37 -0400") References: <525f23e80705130845j52fdb809td3c9de54abcd4aa5@mail.gmail.com> Message-ID: "Alexander Michael" writes: > SVN does support keywords, but you need to turn-on keyword > interpolation (see the SVN documentation describing properties for how > to do this) I know, but the svn $Revision$ keyword doesn't have the semantics I've described (that's what I meant with 'working' -- I don't think the implemented behavior is what one would typically desire; to get a subset of what I described in an awkward and roll-you-own fashion one can muck around with the ``svnversion`` command, but I'd rather use some off-the-shelf mucking around). > but I don't think you'll need this functionality. Development release > suffixes can be added with setuptools > (). > When using this technique, the setup.py version string must be the > *next* version (the one under development that is typically your SVN > Trunk). Here's the rub-- it must be in setup.py and you probably want > to give your package access to it so that it can be reported. It > appears that the current idiom for solving this dilemma is to but a > release.py or version.py file in your package, which is sucked up into > the setup.py file with execfile (see > for an > example). Yuck. I don't think this does what I want either -- how is the development release suffix mean to get into release.py? What kid does seems broken -- it just uses the $Rev$ keyword, but's the local file revision, not that of the project as a whole. > You then manually maintain the version number in one place > (the release.py file). I'm happy to manually maintain the version number (i.e. "1.1"), but not the release number for development releases (i.e. the .dev bit in "1.1.dev43"). > Also note that it appears the current trend is to keep __init__.py > files empty (required if you ever want to create a setuptools > namespace package) and create an api.py file with convenient exports. > This is done to avoid running into issues with circular references. I've heard about this new trend to use mypackage.api before, but I didn't get a particularly good rationale -- I can see that this might can be helpful if you want to be able to import ``mypackage.bar`` without being forced to import the the stuff in ``mypackage``, but that would only seem to apply to a few big projects with independent components -- how does it help with circular references? > I would be tempted to but version information my __init__.py file (and only > version information), but this will breakdown if the package ever grows to > become a namespace package. Perhaps avoiding this simplification, however, > is over future-proofing. I dont see mlabwrap becoming a namespace package. If that really happens I'll be happy to force users to change their import statements for having experienced the convenience of not having to type another 4 characters in the past. 'as From pje at telecommunity.com Sun May 13 18:33:15 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 13 May 2007 12:33:15 -0400 Subject: [Distutils] Reverse dependencies In-Reply-To: References: Message-ID: <20070513163128.0CA063A40B0@sparrow.telecommunity.com> At 05:00 PM 5/13/2007 +0100, Martin Aspeli wrote: >But is there any support for such declarative "reverse" dependencies? Sort of, in a very limited way. You'll get a version compatibility error if you depend on say, Plone<3 and your dependency depends on Plone>=3.1, for example. But there's no backtracking to resolve this, so the minute it happens your installation is basically dead until you get rid of the problematic package. From optilude at gmx.net Sun May 13 19:11:30 2007 From: optilude at gmx.net (Martin Aspeli) Date: Sun, 13 May 2007 18:11:30 +0100 Subject: [Distutils] Reverse dependencies In-Reply-To: <20070513163128.0CA063A40B0@sparrow.telecommunity.com> References: <20070513163128.0CA063A40B0@sparrow.telecommunity.com> Message-ID: Phillip J. Eby wrote: > At 05:00 PM 5/13/2007 +0100, Martin Aspeli wrote: >> But is there any support for such declarative "reverse" dependencies? > > Sort of, in a very limited way. You'll get a version compatibility > error if you depend on say, Plone<3 and your dependency depends on > Plone>=3.1, for example. But there's no backtracking to resolve > this, so the minute it happens your installation is basically dead > until you get rid of the problematic package. Comforting. :-) Do you think it'd be a useful feature? I realise it may not be immediately trivial to add. ;) Martin From pje at telecommunity.com Sun May 13 20:10:55 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 13 May 2007 14:10:55 -0400 Subject: [Distutils] Reverse dependencies In-Reply-To: References: <20070513163128.0CA063A40B0@sparrow.telecommunity.com> Message-ID: <20070513180911.5EE8E3A409B@sparrow.telecommunity.com> At 06:11 PM 5/13/2007 +0100, Martin Aspeli wrote: >Phillip J. Eby wrote: > > At 05:00 PM 5/13/2007 +0100, Martin Aspeli wrote: > >> But is there any support for such declarative "reverse" dependencies? > > > > Sort of, in a very limited way. You'll get a version compatibility > > error if you depend on say, Plone<3 and your dependency depends on > > Plone>=3.1, for example. But there's no backtracking to resolve > > this, so the minute it happens your installation is basically dead > > until you get rid of the problematic package. > >Comforting. :-) > >Do you think it'd be a useful feature? I realise it may not be >immediately trivial to add. ;) No kidding. I believe it's potentially NP-complete, which is why I only implemented a non-backtracking version that always goes for the highest installed version of each package. If somebody else wants to take a look at designing a backtracking algorithm, take a look at the WorkingSet.resolve() method in pkg_resources. From optilude at gmx.net Sun May 13 20:17:55 2007 From: optilude at gmx.net (Martin Aspeli) Date: Sun, 13 May 2007 19:17:55 +0100 Subject: [Distutils] Reverse dependencies In-Reply-To: <20070513180911.5EE8E3A409B@sparrow.telecommunity.com> References: <20070513163128.0CA063A40B0@sparrow.telecommunity.com> <20070513180911.5EE8E3A409B@sparrow.telecommunity.com> Message-ID: Phillip J. Eby wrote: > At 06:11 PM 5/13/2007 +0100, Martin Aspeli wrote: >> Phillip J. Eby wrote: >>> At 05:00 PM 5/13/2007 +0100, Martin Aspeli wrote: >>>> But is there any support for such declarative "reverse" dependencies? >>> Sort of, in a very limited way. You'll get a version compatibility >>> error if you depend on say, Plone<3 and your dependency depends on >>> Plone>=3.1, for example. But there's no backtracking to resolve >>> this, so the minute it happens your installation is basically dead >>> until you get rid of the problematic package. >> Comforting. :-) >> >> Do you think it'd be a useful feature? I realise it may not be >> immediately trivial to add. ;) > > No kidding. I believe it's potentially NP-complete, which is why I > only implemented a non-backtracking version that always goes for the > highest installed version of each package. > > If somebody else wants to take a look at designing a backtracking > algorithm, take a look at the WorkingSet.resolve() method in pkg_resources. Well, it may not need to be quite so complex, but it might require some new spelling. The idea would be to separate declarative platform support from general dependency management (a supported platform may not even be a direct dependency, only a "known good" combination). For example, in your.package (the dependency), we could say: supported_platforms=["Plone>=3.0<4", "Zope>=2.10<2.11"] my.app could say something similar; supported_platforms=["Plone>=3.0.1<4", "Zope>=2.10.3<2.11"] You would only need to look for an overlap in the supported platforms. It may be prudent to assume that if a platform wasn't specified, it's still OK, i.e. don't fail if you don't know. Martin From a.schmolck at gmx.net Sun May 13 23:00:23 2007 From: a.schmolck at gmx.net (Alexander Schmolck) Date: Sun, 13 May 2007 22:00:23 +0100 Subject: [Distutils] unpacking files from and egg only for testing In-Reply-To: <20070513162117.C061E3A40B0@sparrow.telecommunity.com> (Phillip J. Eby's message of "Sun\, 13 May 2007 12\:23\:04 -0400") References: <20070513162117.C061E3A40B0@sparrow.telecommunity.com> Message-ID: "Phillip J. Eby" writes: > Use pkg_resources.resource_filename() - it will give you a real filename for > the resource (unpacked to a temporary directory if necessary). If you are > running "setup.py test", on the other hand, it will return the path to the > original source file. Great exactly what I was looking for -- thanks. cheers, 'as From pje at telecommunity.com Mon May 14 00:17:06 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 13 May 2007 18:17:06 -0400 Subject: [Distutils] Reverse dependencies In-Reply-To: References: <20070513163128.0CA063A40B0@sparrow.telecommunity.com> <20070513180911.5EE8E3A409B@sparrow.telecommunity.com> Message-ID: <20070513221524.D711D3A409B@sparrow.telecommunity.com> At 07:17 PM 5/13/2007 +0100, Martin Aspeli wrote: >Phillip J. Eby wrote: > > At 06:11 PM 5/13/2007 +0100, Martin Aspeli wrote: > >> Phillip J. Eby wrote: > >>> At 05:00 PM 5/13/2007 +0100, Martin Aspeli wrote: > >>>> But is there any support for such declarative "reverse" dependencies? > >>> Sort of, in a very limited way. You'll get a version compatibility > >>> error if you depend on say, Plone<3 and your dependency depends on > >>> Plone>=3.1, for example. But there's no backtracking to resolve > >>> this, so the minute it happens your installation is basically dead > >>> until you get rid of the problematic package. > >> Comforting. :-) > >> > >> Do you think it'd be a useful feature? I realise it may not be > >> immediately trivial to add. ;) > > > > No kidding. I believe it's potentially NP-complete, which is why I > > only implemented a non-backtracking version that always goes for the > > highest installed version of each package. > > > > If somebody else wants to take a look at designing a backtracking > > algorithm, take a look at the WorkingSet.resolve() method in > pkg_resources. > >Well, it may not need to be quite so complex, but it might require some >new spelling. The idea would be to separate declarative platform support >from general dependency management (a supported platform may not even be >a direct dependency, only a "known good" combination). > >For example, in your.package (the dependency), we could say: > > supported_platforms=["Plone>=3.0<4", "Zope>=2.10<2.11"] > >my.app could say something similar; > > supported_platforms=["Plone>=3.0.1<4", "Zope>=2.10.3<2.11"] > >You would only need to look for an overlap in the supported platforms. I don't see how this is different from what I said. It's still a dependency (and dependency conflict), and it requires the same kind of backtracking to support handling it. Thus, I don't see any reason to create another kind of dependency for it, because it wouldn't actually simplify anything as far as I can tell. From optilude at gmx.net Mon May 14 00:30:27 2007 From: optilude at gmx.net (Martin Aspeli) Date: Sun, 13 May 2007 23:30:27 +0100 Subject: [Distutils] Reverse dependencies In-Reply-To: <20070513221524.D711D3A409B@sparrow.telecommunity.com> References: <20070513163128.0CA063A40B0@sparrow.telecommunity.com> <20070513180911.5EE8E3A409B@sparrow.telecommunity.com> <20070513221524.D711D3A409B@sparrow.telecommunity.com> Message-ID: Phillip J. Eby wrote: > At 07:17 PM 5/13/2007 +0100, Martin Aspeli wrote: >> Phillip J. Eby wrote: >>> At 06:11 PM 5/13/2007 +0100, Martin Aspeli wrote: >>>> Phillip J. Eby wrote: >>>>> At 05:00 PM 5/13/2007 +0100, Martin Aspeli wrote: >>>>>> But is there any support for such declarative "reverse" dependencies? >>>>> Sort of, in a very limited way. You'll get a version compatibility >>>>> error if you depend on say, Plone<3 and your dependency depends on >>>>> Plone>=3.1, for example. But there's no backtracking to resolve >>>>> this, so the minute it happens your installation is basically dead >>>>> until you get rid of the problematic package. >>>> Comforting. :-) >>>> >>>> Do you think it'd be a useful feature? I realise it may not be >>>> immediately trivial to add. ;) >>> No kidding. I believe it's potentially NP-complete, which is why I >>> only implemented a non-backtracking version that always goes for the >>> highest installed version of each package. >>> >>> If somebody else wants to take a look at designing a backtracking >>> algorithm, take a look at the WorkingSet.resolve() method in >> pkg_resources. >> >> Well, it may not need to be quite so complex, but it might require some >> new spelling. The idea would be to separate declarative platform support >>from general dependency management (a supported platform may not even be >> a direct dependency, only a "known good" combination). >> >> For example, in your.package (the dependency), we could say: >> >> supported_platforms=["Plone>=3.0<4", "Zope>=2.10<2.11"] >> >> my.app could say something similar; >> >> supported_platforms=["Plone>=3.0.1<4", "Zope>=2.10.3<2.11"] >> >> You would only need to look for an overlap in the supported platforms. > > I don't see how this is different from what I said. It's still a > dependency (and dependency conflict), and it requires the same kind > of backtracking to support handling it. Thus, I don't see any reason > to create another kind of dependency for it, because it wouldn't > actually simplify anything as far as I can tell. I won't pretend to have thought this through to very end, but I don't quite understand why this should be so hard, then. The algorithm would be something like: - I support platform Plone, version >= 3.0.1 and < 4 - I am looking for package your.package, version >= 1.0 - The most recent version that satisfies this is 2.1 - Version 2.1 declares it supports Plone >=4, can't use that - The second-most-recent version that satisfies your.package >= 1.0 is 2.0 - Version 2.0 declares it supports Plone 3.9, okay, we can use that - Repeat check for Zope Actually... I see how it could get complicated with lots of platform dependencies. :) I wonder if this can't be simplified to the common case where there's only one "main" platform to worry about, though, e.g. only check "Plone" from my.app in the case above, but let your.package declare support for various "known good" configurations, such as "with Plone 3" and "with Zope 2.10". One difference to straight dependencies, by the way, is the case where you support (or a package is known to work with) a platform, without necessarily needing to depend on it. That may be useful metadata. I'm mostly just thinking out loud, though. :) Martin From pje at telecommunity.com Mon May 14 01:45:32 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 13 May 2007 19:45:32 -0400 Subject: [Distutils] Reverse dependencies In-Reply-To: References: <20070513163128.0CA063A40B0@sparrow.telecommunity.com> <20070513180911.5EE8E3A409B@sparrow.telecommunity.com> <20070513221524.D711D3A409B@sparrow.telecommunity.com> Message-ID: <20070513234346.D3F563A409B@sparrow.telecommunity.com> At 11:30 PM 5/13/2007 +0100, Martin Aspeli wrote: >I won't pretend to have thought this through to very end, but I don't >quite understand why this should be so hard, then. > >The algorithm would be something like: > > - I support platform Plone, version >= 3.0.1 and < 4 > - I am looking for package your.package, version >= 1.0 > - The most recent version that satisfies this is 2.1 > - Version 2.1 declares it supports Plone >=4, can't use that > - The second-most-recent version that satisfies > your.package >= 1.0 is 2.0 > - Version 2.0 declares it supports Plone 3.9, okay, we can use that > > - Repeat check for Zope This isn't actually any different than a backtracking dependency search. The only thing that makes it (potentially) simpler is that it could simply allow backing off only for immediate dependencies, rather than backtracking transitive dependencies. That is, while selecting the "best version available" of a package, we could check its immediate dependencies for conflict with all known dependencies, and automatically back off. Practically speaking, however, it's still considerably more complex than the existing algorithm -- and still not certain to produce valid results. >One difference to straight dependencies, by the way, is the case where >you support (or a package is known to work with) a platform, without >necessarily needing to depend on it. That may be useful metadata. Useful for what? From optilude at gmx.net Mon May 14 01:57:59 2007 From: optilude at gmx.net (Martin Aspeli) Date: Mon, 14 May 2007 00:57:59 +0100 Subject: [Distutils] Reverse dependencies In-Reply-To: <20070513234346.D3F563A409B@sparrow.telecommunity.com> References: <20070513163128.0CA063A40B0@sparrow.telecommunity.com> <20070513180911.5EE8E3A409B@sparrow.telecommunity.com> <20070513221524.D711D3A409B@sparrow.telecommunity.com> <20070513234346.D3F563A409B@sparrow.telecommunity.com> Message-ID: Phillip J. Eby wrote: > At 11:30 PM 5/13/2007 +0100, Martin Aspeli wrote: >> I won't pretend to have thought this through to very end, but I don't >> quite understand why this should be so hard, then. >> >> The algorithm would be something like: >> >> - I support platform Plone, version >= 3.0.1 and < 4 >> - I am looking for package your.package, version >= 1.0 >> - The most recent version that satisfies this is 2.1 >> - Version 2.1 declares it supports Plone >=4, can't use that >> - The second-most-recent version that satisfies >> your.package >= 1.0 is 2.0 >> - Version 2.0 declares it supports Plone 3.9, okay, we can use that >> >> - Repeat check for Zope > > This isn't actually any different than a backtracking dependency > search. The only thing that makes it (potentially) simpler is that > it could simply allow backing off only for immediate dependencies, > rather than backtracking transitive dependencies. > > That is, while selecting the "best version available" of a package, > we could check its immediate dependencies for conflict with all known > dependencies, and automatically back off. > > Practically speaking, however, it's still considerably more complex > than the existing algorithm -- and still not certain to produce valid results. Yes, I think you're right. >> One difference to straight dependencies, by the way, is the case where >> you support (or a package is known to work with) a platform, without >> necessarily needing to depend on it. That may be useful metadata. > > Useful for what? Useful if the above algorithm can be made to work: I have a package which does not directly require Plone (so I don't want this installed as a dependency), say, but which I know works with Plone 3, and not with Plone 2.5 (because of other things changing in Plone). In this case, I can usually signal this to other packages which wants "the version of this package which is known to work with the platform I'm using". Martin From a.schmolck at gmx.net Mon May 14 02:09:01 2007 From: a.schmolck at gmx.net (Alexander Schmolck) Date: Mon, 14 May 2007 01:09:01 +0100 Subject: [Distutils] automating __version__' ing issues In-Reply-To: <20070513162836.041E23A40B0@sparrow.telecommunity.com> (Phillip J. Eby's message of "Sun\, 13 May 2007 12\:30\:21 -0400") References: <20070513162836.041E23A40B0@sparrow.telecommunity.com> Message-ID: "Phillip J. Eby" writes: >>Here's the rub-- it must be in setup.py and you probably want >>to give your package access to it so that it can be reported. It >>appears that the current idiom for solving this dilemma is to but a >>release.py or version.py file in your package, which is sucked up into >>the setup.py file with execfile (see >> for an >>example). You then manually maintain the version number in one place >>(the release.py file). > > Yep, this is what most people do. Is there a standard trick to get the desired svn-revision? Running `'svnversion > __svn_version__.py`` in setup.py when a distrubtion-building command is issued and then importing that in a hand-maintained ``version.py``/``release.py`` file should work, but is there a way to check whether a setup.py command falls into a certain category? > I actually use a PEAK tool called "version" (peak.tools.version) which > maintains a data file with the version, and does targeted search-and-replace > operations on the files listed in a configuration file. This is a lot easier > when you have documentation files that also need version information > updated, and it also avoids the execfile pain. This sounds more like it, but I don't assume this package is going to be ready for an official release anytime soon? cheers, 'as From pje at telecommunity.com Mon May 14 02:39:20 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 13 May 2007 20:39:20 -0400 Subject: [Distutils] automating __version__' ing issues In-Reply-To: References: <20070513162836.041E23A40B0@sparrow.telecommunity.com> Message-ID: <20070514003734.7EF9C3A409B@sparrow.telecommunity.com> At 01:09 AM 5/14/2007 +0100, Alexander Schmolck wrote: >"Phillip J. Eby" writes: > > >>Here's the rub-- it must be in setup.py and you probably want > >>to give your package access to it so that it can be reported. It > >>appears that the current idiom for solving this dilemma is to but a > >>release.py or version.py file in your package, which is sucked up into > >>the setup.py file with execfile (see > >> for an > >>example). You then manually maintain the version number in one place > >>(the release.py file). > > > > Yep, this is what most people do. > >Is there a standard trick to get the desired svn-revision? Running >`'svnversion > __svn_version__.py`` in setup.py when a distrubtion-building >command is issued and then importing that in a hand-maintained >``version.py``/``release.py`` file should work, but is there a way to check >whether a setup.py command falls into a certain category? I don't understand what you're trying to do. Why do you need the SVN revision in a .py file? > > I actually use a PEAK tool called "version" (peak.tools.version) which > > maintains a data file with the version, and does targeted > search-and-replace > > operations on the files listed in a configuration file. This is a > lot easier > > when you have documentation files that also need version information > > updated, and it also avoids the execfile pain. > >This sounds more like it, but I don't assume this package is going to be ready >for an official release anytime soon? Nope, and it's not documented either. I stole the idea, however, from another open source program that does something quite similar, but unfortunately I don't remember the name of that other program. From peter.mowry at amd.com Mon May 14 04:12:15 2007 From: peter.mowry at amd.com (Mowry, Peter) Date: Sun, 13 May 2007 21:12:15 -0500 Subject: [Distutils] Python module to: Talk to stdout/stdin for 64-bit Windows/Linux C++ application? In-Reply-To: <15086.1179025179@d.0x1.org> References: Your message of "Sat, 12 May 2007 19:09:55 EST." <475F4CA05E4E1B46A7213674E70EFA62022968CA@SAUSEXMB2.amd.com> <15086.1179025179@d.0x1.org> Message-ID: <475F4CA05E4E1B46A7213674E70EFA6202296938@SAUSEXMB2.amd.com> David, The subprocess module doesn't work for interactive programs (I think) (see appended pexpect faq). So instead, I was trying to get something like the following working: import simnow # import complex simnow C++ application simnow.start() # spawn simnow's main method output1 = simnow.sendCommand("load platform x") # blocking call to a single simnow C++ functions # This is our real current goal, but if I get this working, we might consider exposing other simnow functionality to the simnow python module. disutils is listed in the extend/embed python doc (http://www.python.org/doc/ext/ext.html). This doc seems to be saying that disutils is a cross-plaform way to build a python module ("extend python"). Chapter 3 - disultils cross-platform build. Chapter 4 - do it yourself windows build. Chapter 5 - do it yourself unix build. Chapter 4 says, "Module authors are encouraged to use the distutils approach for building extension modules, instead of the one described in this section." --Thanks subprocess - "appended comments" # This doesn't work self.simnow = Popen(executable=(os.getcwd() + '/simnow'), args=simnowCmd, stdin=PIPE, stdout=PIPE, stderr=PIPE) self.simnow.stdin.write("open stuff") print self.simnow.stdout.read() Why doesn't it work? The following is from the pexect FAQ Q: Why not just use a pipe (popen())? A: A pipe works fine for getting the output to non-interactive programs. If you just want to get the output from ls, uname, or ping then this works. Pipes do not work very well for interactive programs and pipes will almost certainly fail for most applications that ask for passwords such as telnet, ftp, or ssh. There are two reasons for this. First an application may bypass stdout and print directly to its controlling TTY. Something like SSH will do this when it asks you for a password. This is why you cannot redirect the password prompt because it does not go through stdout or stderr. The second reason is because most applications are built using the C Standard IO Library (anything that uses #include ). One of the features of the stdio library is that it buffers all input and output. Normally output is line buffered when a program is printing to a TTY (your terminal screen). Everytime the program prints a line-feed the currently buffered data will get printed to your screen. The problem comes when you connect a pipe. The stdio library is smart and can tell that it is printing to a pipe instead of a TTY. In that case it switches from line buffer mode to block buffered. In this mode the currently buffered data is flushed when the buffer is full. This causes most interactive programs to deadlock. Block buffering is more efficient when writing to disks and pipes. Take the situation where a program prints a message "Enter your user name:\n" and then waits for you type type something. In block buffered mode, the stdio library will not put the message into the pipe even though a linefeed is printed. The result is that you never receive the message, yet the child application will sit and wait for you to type a response. Don't confuse the stdio lib's buffer with the pipe's buffer. The pipe buffer is another area that can cause problems. You could flush the input side of a pipe, whereas you have no control over the stdio library buffer. http://pexpect.sourceforge.net/#faq What's wrong with pexpect? It doesn't work on Windows. Python COM is an alternative, but we are trying to find a clean unified cross-platform solution. Peter Mowry ("Pem") SimNow Team, Software Engineer AMD Organization -----Original Message----- From: David Arnold [mailto:david at mantara.com] Sent: Saturday, May 12, 2007 10:00 PM To: Mowry, Peter Cc: distutils-sig at python.org Subject: Re: [Distutils] Python module to: Talk to stdout/stdin for 64-bit Windows/Linux C++ application? -->"Peter" == Mowry, Peter writes: Peter> My team's application does regressions by controlling our Peter> 64-bit Windows/Linux C++ application (SimNow, Peter> www.amd.com/simnow) via stdout/stdin with ("pexpect" for unix) Peter> and (python COM objects) for Windows (b/c popen doesn't work). Peter> However, I am trying to replace this with a unified Peter> windows/linux solution. have you looked at the subprocess module, part of the standard library in Python 2.4 and later? Peter> Does extending python with Disutils sound like the simple and Peter> clean way for my goal? Is there a simple cookbook solution for Peter> my goal? i don't think distutils is what you're after. distutils is about producing installers for Python applications. d -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20070513/56da1461/attachment-0001.htm From matt at matt-good.net Mon May 14 05:53:19 2007 From: matt at matt-good.net (Matt Good) Date: Sun, 13 May 2007 20:53:19 -0700 Subject: [Distutils] Python module to: Talk to stdout/stdin for 64-bit Windows/Linux C++ application? In-Reply-To: <475F4CA05E4E1B46A7213674E70EFA6202296938@SAUSEXMB2.amd.com> References: Your message of "Sat, 12 May 2007 19:09:55 EST." <475F4CA05E4E1B46A7213674E70EFA62022968CA@SAUSEXMB2.amd.com> <15086.1179025179@d.0x1.org> <475F4CA05E4E1B46A7213674E70EFA6202296938@SAUSEXMB2.amd.com> Message-ID: <4647DD2F.3000706@matt-good.net> Mowry, Peter wrote: > > David, > > > > The subprocess module doesn't work for interactive programs (I think) > (see appended pexpect faq). > > > > So instead, I was trying to get something like the following working: > > import simnow # import complex simnow C++ application > > simnow.start() # spawn simnow's main method > > output1 = simnow.sendCommand("load platform x") # blocking call to a > single simnow C++ functions > > # This is our real current goal, but if I get this working, we might > consider exposing other simnow functionality to the simnow python module. > > > > disutils is listed in the extend/embed python doc > (http://www.python.org/doc/ext/ext.html). This doc seems to be saying > that disutils is a cross-plaform way to build a python module ("extend > python"). > > > Distutils is kind of like "make" for Python. It's there to help you build and distribute Python modules, but that's basically all it does. There's nothing in distutils to help with cross-platform process IO. Based on the pexpect page it looks like it should work with Cygwin on Windows, so you could consider using that instead. -- Matt Good From jim at zope.com Mon May 14 12:58:58 2007 From: jim at zope.com (Jim Fulton) Date: Mon, 14 May 2007 06:58:58 -0400 Subject: [Distutils] Reverse dependencies In-Reply-To: <20070513180911.5EE8E3A409B@sparrow.telecommunity.com> References: <20070513163128.0CA063A40B0@sparrow.telecommunity.com> <20070513180911.5EE8E3A409B@sparrow.telecommunity.com> Message-ID: FWIW, I expect to have to implement this for buildout at some point. I don't plan to do it until it becomes a priority. :) WRT platform, I see this as an anti-solution. Fortunately, it looks like it has already been dismissed. :) The reason I see this as an anti-solution is that we already have a platform, namely Python 2.4, that might be better treated as a dependency. For example, there isn't a way, AFAIK, to state Python version requirements for source releases. I think it might be cleaner if things like Python version and maybe even size of unicode characters could be handled using the dependency mechanism. Jim On May 13, 2007, at 2:10 PM, Phillip J. Eby wrote: > At 06:11 PM 5/13/2007 +0100, Martin Aspeli wrote: >> Phillip J. Eby wrote: >>> At 05:00 PM 5/13/2007 +0100, Martin Aspeli wrote: >>>> But is there any support for such declarative "reverse" >>>> dependencies? >>> >>> Sort of, in a very limited way. You'll get a version compatibility >>> error if you depend on say, Plone<3 and your dependency depends on >>> Plone>=3.1, for example. But there's no backtracking to resolve >>> this, so the minute it happens your installation is basically dead >>> until you get rid of the problematic package. >> >> Comforting. :-) >> >> Do you think it'd be a useful feature? I realise it may not be >> immediately trivial to add. ;) > > No kidding. I believe it's potentially NP-complete, which is why I > only implemented a non-backtracking version that always goes for the > highest installed version of each package. > > If somebody else wants to take a look at designing a backtracking > algorithm, take a look at the WorkingSet.resolve() method in > pkg_resources. > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig -- Jim Fulton mailto:jim at zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From torfer at pvv.org Mon May 14 13:36:55 2007 From: torfer at pvv.org (Tore Ferner) Date: Mon, 14 May 2007 13:36:55 +0200 Subject: [Distutils] stdeb bug: syntax error during dpkg -i In-Reply-To: <46443FDF.2010104@astraw.com> References: <464232C3.7030300@pvv.org> <4643A59C.3010307@astraw.com> <46443FDF.2010104@astraw.com> Message-ID: <464849D7.2040702@pvv.org> Andrew Straw wrote: [snip] > Hmm, this is strange. Apparently pycentral isn't interested in > Python2.3, and yet it does get called. It's possible there's a pycentral > bug, but I'm not sure. Can you post the contents of your > deb-dst/bzr-0.16.0/debian/rules file? Hi, It's at the end of the mail. > I don't have Python2.3 installed, > so I'm flying blindly here... > >> >>> FWIW, I built a version of python-bzr_0.16.0-1_all.deb using >>> more-or-less the commands you issued just now on Ubuntu Edgy. Upon >>> installing it, all went well, and I didn't get the issues you're having. >>> >>> Also, I just added a section "A complete example" to the webpage >>> http://stdeb.python-hosting.com/ using your package as an example. This >>> works fine on my system. >>> >> >> Yes, that's helpful. Thank you! >> >> I have a couple of questions; if you have time, maybe you could add >> information about this to your on-line example?: >> >> 1) bzr is, I suppose, a standalone program rather than a python module. >> It would be proper to name the package "bzr", "bzr-local", "bzr-mybuild" >> or something. I think "bzr" would be best, or perhaps that it at least >> provides bzr. >> What would be the best way to achieve this with stdeb? >> > Hmm, I'll have to add this. This shouldn't hard. >> 2) Would it be possible to get the source documentation under doc/ >> placed in /usr/share/doc/package-name (e.g. /usr/share/doc/python-bzr)? >> > Again, this is another feature request -- I'll have to add support for > it. Unfortunately, there's not a universal standard for Python > documentation (separate from the source code), so this one will require > some thinking about. >> 3) How to shift the installation from /usr to /usr/local? (? la >> --prefix=/usr/local) >> > Hmm, another good question... Does pycentral actually support that? I > guess it does. OK, I'll consider this another feature request! :) > > I'll send an email when I've implemented any or all of these features... That sounds great! Best regards Tore Rules file: #!/usr/bin/make -f # This file was automatically generated by stdeb 0.2.a1 at # Wed, 9 May 2007 22:35:31 +0200 PACKAGE_NAME=python-bzr MODULE_NAME=bzr DEB_UPSTREAM_VERSION=0.16.0 PYVERS=$(shell pyversions -vr) build: build-stamp build-stamp: $(PYVERS:%=build-python%) touch $@ build-python%: # Force setuptools, but reset sys.argv[0] to 'setup.py' because setup.py files expect that. python$* -c "import setuptools,sys;f='setup.py';sys.argv[0]=f;execfile(f,{'__file__':f,'__name__':'__main__'})" build touch $@ clean: dh_testdir dh_testroot rm -f *-stamp rm -rf dist build -find -name '*.py[co]' | xargs rm -f # find . -name *.pyc -exec rm {} \; dh_clean install: build install-prereq $(PYVERS:%=install-python%) install-prereq: dh_testdir dh_testroot dh_clean -k install-python%: # Force setuptools, but reset sys.argv[0] to 'setup.py' because setup.py files expect that. python$* -c "import setuptools,sys;f='setup.py';sys.argv[0]=f;execfile(f,{'__file__':f,'__name__':'__main__'})" install --no-compile --single-version-externally-managed --root $(CURDIR)/debian/${PACKAGE_NAME} mv debian/${PACKAGE_NAME}/usr/lib/python$*/site-packages/${MODULE_NAME}-${DEB_UPSTREAM_VERSION}-py$*.egg-info debian/${PACKAGE_NAME}/usr/lib/python$*/site-packages/${MODULE_NAME}.egg-info binary-arch: binary-indep: build install dh_testdir -i dh_testroot -i dh_pycentral -i dh_installdocs -i dh_installdirs -i dh_installexamples -i dh_strip -i dh_compress -i -X.py dh_fixperms -i : # Replace all '#!' calls to python with $(PYTHON) : # and make them executable for i in \ `find debian/python-bzr/usr/bin -type f` \ `find debian/python-bzr/usr/lib -type f`; \ do \ case "$$i" in *-[0-9].[0-9]) continue; esac; \ sed '1s,#!.*python[^ ]*\(.*\),#! /usr/bin/python\1,' \ $$i > $$i.temp; \ if cmp --quiet $$i $$i.temp; then \ rm -f $$i.temp; \ else \ mv -f $$i.temp $$i; \ chmod 755 $$i; \ echo "fixed interpreter: $$i"; \ fi; \ done dh_installdeb -i dh_gencontrol -i dh_md5sums -i dh_builddeb -i binary: binary-indep binary-arch .PHONY: build clean binary-indep binary-arch binary install configure From a.schmolck at gmx.net Mon May 14 14:58:10 2007 From: a.schmolck at gmx.net (Alexander Schmolck) Date: Mon, 14 May 2007 13:58:10 +0100 Subject: [Distutils] automating __version__' ing issues In-Reply-To: <20070514003734.7EF9C3A409B@sparrow.telecommunity.com> (Phillip J. Eby's message of "Sun\, 13 May 2007 20\:39\:20 -0400") References: <20070514003734.7EF9C3A409B@sparrow.telecommunity.com> Message-ID: "Phillip J. Eby" writes: > At 01:09 AM 5/14/2007 +0100, Alexander Schmolck wrote: >>"Phillip J. Eby" writes: >> >> >>Here's the rub-- it must be in setup.py and you probably want >> >>to give your package access to it so that it can be reported. It >> >>appears that the current idiom for solving this dilemma is to but a >> >>release.py or version.py file in your package, which is sucked up into >> >>the setup.py file with execfile (see >> >> for an >> >>example). You then manually maintain the version number in one place >> >>(the release.py file). >> > >> > Yep, this is what most people do. >> >>Is there a standard trick to get the desired svn-revision? Running >>`'svnversion > __svn_version__.py`` in setup.py when a distrubtion-building >>command is issued and then importing that in a hand-maintained >>``version.py``/``release.py`` file should work, but is there a way to check >>whether a setup.py command falls into a certain category? > > I don't understand what you're trying to do. I'd like to have a mechanism that has me specify two things in *one* central location: 1. A self-chosen version number, e.g. "1.1" 2. A release flag (e.g. official_release=False, meaning it's an official and not a development release) Now in all other locations I'd like to get a version string that either is "1.1" if release is True "1.1.dev43" (or whatever the svn revision) if release is False. All other locations means at least setup.py and a __init__.py (via import or execfile if needs be). > Why do you need the SVN revision in a .py file? I need it somewhere where it can be accessed by setup.py and the installed package at a negligible cost. Using some convoluted invocation to create a __svn_version__.py file in setup.py is the way numpy.distutils based project do it, but I'd really like to avoid that (and numpy.distutils in general). >>This sounds more like it, but I don't assume this package is going to be ready >>for an official release anytime soon? > > Nope, and it's not documented either. I stole the idea, however, from another > open source program that does something quite similar, but unfortunately I > don't remember the name of that other program. vertoo? Unfortunately that's alpha, too (and no releases since 2003). cheers, 'as p.s. I notice that a cc: or to: to your email address bounces -- is that intended? (For what it's worth, I'd prefer if docutils-sig would just set the reply to header to the list anyway -- it's what the scipy, numpy and many other lists already do anyway and clearly the desired default behavior for the vast majority of replies to messages on this or most technical lists; I guess it might some unfortunate standard for the *-sig lists, though). From jim at zope.com Mon May 14 15:52:45 2007 From: jim at zope.com (Jim Fulton) Date: Mon, 14 May 2007 09:52:45 -0400 Subject: [Distutils] setuptools process_url doesn't handle URLErrors gracefullly In-Reply-To: <1178871702.22778.8.camel@mindy.whq.gocept.com> References: <1178871702.22778.8.camel@mindy.whq.gocept.com> Message-ID: This problem is rather serious. As we rely more and more on setuptools, it needs to be robust in the presence of this sort of failure. Phillip, If I fix this, can we get a timely release with the fix? (Otherwise, I'll work on a fix in buildout.) Jim On May 11, 2007, at 4:21 AM, Christian Theune wrote: > Hi, > > I'm fighting against a problem where the DNS name of the homepage > of a > package that is listed on the cheeseshop isn't accessible any more. In > this special case the DNS name for this thing is gone (the package is > `rwproperty` the missing DNS entry is z3lab.org). > > In this case easy_install dies with the following error, although the > eggs are listed on the cheeseshop page and the listed homepage is > pretty > irrelevant: > > Searching for rwproperty > Reading http://cheeseshop.python.org/pypi/rwproperty/ > Reading > http://www.z3lab.org/sections/blogs/philipp-weitershausen/ > 2006_05_29_pycon-06-lightning-talk > Traceback (most recent call last): > File "/usr/bin/easy_install", line 7, in ? > sys.exit( > File > "/usr/lib/python2.4/site-packages/setuptools-0.6c5-py2.4.egg/ > setuptools/command/easy_install.py", line 1670, in main > with_ei_usage(lambda: > File > "/usr/lib/python2.4/site-packages/setuptools-0.6c5-py2.4.egg/ > setuptools/command/easy_install.py", line 1659, in with_ei_usage > return f() > File > "/usr/lib/python2.4/site-packages/setuptools-0.6c5-py2.4.egg/ > setuptools/command/easy_install.py", line 1674, in > distclass=DistributionWithoutHelpCommands, **kw > File "/usr/lib/python2.4/distutils/core.py", line 149, in setup > dist.run_commands() > File "/usr/lib/python2.4/distutils/dist.py", line 946, in > run_commands > self.run_command(cmd) > File "/usr/lib/python2.4/distutils/dist.py", line 966, in > run_command > cmd_obj.run() > File > "/usr/lib/python2.4/site-packages/setuptools-0.6c5-py2.4.egg/ > setuptools/command/easy_install.py", line 211, in run > self.easy_install(spec, not self.no_deps) > File > "/usr/lib/python2.4/site-packages/setuptools-0.6c5-py2.4.egg/ > setuptools/command/easy_install.py", line 432, in easy_install > dist = self.package_index.fetch_distribution( > File > "/usr/lib/python2.4/site-packages/setuptools-0.6c5-py2.4.egg/ > setuptools/package_index.py", line 462, in fetch_distribution > self.find_packages(requirement) > File > "/usr/lib/python2.4/site-packages/setuptools-0.6c5-py2.4.egg/ > setuptools/package_index.py", line 303, in find_packages > self.scan_url(self.index_url + requirement.unsafe_name+'/') > File > "/usr/lib/python2.4/site-packages/setuptools-0.6c5-py2.4.egg/ > setuptools/package_index.py", line 610, in scan_url > self.process_url(url, True) > File > "/usr/lib/python2.4/site-packages/setuptools-0.6c5-py2.4.egg/ > setuptools/package_index.py", line 201, in process_url > page = self.process_index(url, page) > File > "/usr/lib/python2.4/site-packages/setuptools-0.6c5-py2.4.egg/ > setuptools/package_index.py", line 278, in process_index > self.scan_url(new_url) > File > "/usr/lib/python2.4/site-packages/setuptools-0.6c5-py2.4.egg/ > setuptools/package_index.py", line 610, in scan_url > self.process_url(url, True) > File > "/usr/lib/python2.4/site-packages/setuptools-0.6c5-py2.4.egg/ > setuptools/package_index.py", line 191, in process_url > self.fetched_urls[url] = self.fetched_urls[f.url] = True > AttributeError: URLError instance has no attribute 'url' > > I didn't find any switch to turn to make this go away and I don't know > the setuptools code so my few feeble attempts to fix this didn't work > out. > > Some help would be very appreciated. > > Christian > > > -- > gocept gmbh & co. kg - forsterstra?e 29 - 06112 halle/saale - germany > www.gocept.com - ct at gocept.com - phone +49 345 122 9889 7 - > fax +49 345 122 9889 1 - zope and plone consulting and development > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig -- Jim Fulton mailto:jim at zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From optilude at gmx.net Mon May 14 21:02:45 2007 From: optilude at gmx.net (Martin Aspeli) Date: Mon, 14 May 2007 20:02:45 +0100 Subject: [Distutils] Reverse dependencies In-Reply-To: References: <20070513163128.0CA063A40B0@sparrow.telecommunity.com> <20070513180911.5EE8E3A409B@sparrow.telecommunity.com> Message-ID: Jim Fulton wrote: > FWIW, I expect to have to implement this for buildout at some point. > I don't plan to do it until it becomes a priority. :) Interesting. :) > WRT platform, I see this as an anti-solution. Fortunately, it looks > like it has already been dismissed. :) The reason I see this as an > anti-solution is that we already have a platform, namely Python 2.4, > that might be better treated as a dependency. For example, there > isn't a way, AFAIK, to state Python version requirements for source > releases. I think it might be cleaner if things like Python version > and maybe even size of unicode characters could be handled using the > dependency mechanism. To me, the word "platform" means more than Python version and low-level OS things. I also meant things like "Plone" or "Zope". I'm not saying these can't be handled by dependency backtracking, but it may be more practical to think in terms of some higher level constructs. For example, if I develop a package for Plone, I may target Plone 3, test with Plone 3.0.x and know that it works with 3.0.1. However, I may not have any direct dependencies on Plone - the package could be more general, but other things used in Plone may mean that I know it works with 3.0.1 but no earlier versions, and possibly not 4.0+ either. In a perfect world, entirely strict dependencies should tell us all this. However, in practice, since things like Plone and Zope are released as whole units, and not necessarily managed strictly as a system of with eggs requiring each other (which, for something like Plone, could end up being quite complex to manage), that may not work. The algorithm to find them could be pretty hairy as well. As the builder of an application on top of Plone, all I really want is "the version of the package that works with Plone 3.0.2" (or whichever version I'm using). In this case, such "known-good platform" metadata may be quite useful. Martin From jim at zope.com Mon May 14 21:11:59 2007 From: jim at zope.com (Jim Fulton) Date: Mon, 14 May 2007 15:11:59 -0400 Subject: [Distutils] Reverse dependencies In-Reply-To: References: <20070513163128.0CA063A40B0@sparrow.telecommunity.com> <20070513180911.5EE8E3A409B@sparrow.telecommunity.com> Message-ID: On May 14, 2007, at 3:02 PM, Martin Aspeli wrote: > Jim Fulton wrote: >> FWIW, I expect to have to implement this for buildout at some point. >> I don't plan to do it until it becomes a priority. :) > > Interesting. :) Hey, I have lots of things to do. I have to prioritize. :) BTW, please don't request this until you really need it. >> WRT platform, I see this as an anti-solution. Fortunately, it looks >> like it has already been dismissed. :) The reason I see this as an >> anti-solution is that we already have a platform, namely Python 2.4, >> that might be better treated as a dependency. For example, there >> isn't a way, AFAIK, to state Python version requirements for source >> releases. I think it might be cleaner if things like Python version >> and maybe even size of unicode characters could be handled using the >> dependency mechanism. > > To me, the word "platform" means more than Python version and low- > level > OS things. I also meant things like "Plone" or "Zope". I'm not saying > these can't be handled by dependency backtracking, but it may be more > practical to think in terms of some higher level constructs. > > For example, if I develop a package for Plone, I may target Plone 3, > test with Plone 3.0.x and know that it works with 3.0.1. > > However, I may not have any direct dependencies on Plone - the package > could be more general, but other things used in Plone may mean that I > know it works with 3.0.1 but no earlier versions, and possibly not > 4.0+ > either. > > In a perfect world, entirely strict dependencies should tell us all > this. However, in practice, since things like Plone and Zope are > released as whole units, and not necessarily managed strictly as a > system of with eggs requiring each other (which, for something like > Plone, could end up being quite complex to manage), that may not work. > The algorithm to find them could be pretty hairy as well. > > As the builder of an application on top of Plone, all I really want is > "the version of the package that works with Plone 3.0.2" (or whichever > version I'm using). In this case, such "known-good platform" metadata > may be quite useful. I was just trying to note in a round about way that I prefer the simpler model of dependencies. We don't need to introduce a new concept. We just need a better version-resolution algorithm. We're going to need this sooner or later regardless of whether we have anything that someone might choose to call a platform. Note that having such an algorithm is more practical for buildout than with easy_install because the resolution is done at install time, rather than at run time, and can thus take longer if it needs to. jim -- Jim Fulton mailto:jim at zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From jmg3000 at gmail.com Mon May 14 22:01:07 2007 From: jmg3000 at gmail.com (John Gabriele) Date: Mon, 14 May 2007 16:01:07 -0400 Subject: [Distutils] keeping a local ~/pylib directory of installed modules Message-ID: <65e0bb520705141301l4742e45ei469c6839c5370709@mail.gmail.com> Hi, I'm running on a server where I don't have root access. I've installed my own Python into ~/py-2.5.1 (with a ~/py symlink), and created my own ~/pylib directory for installing any extra packages I need. I'd like to start using Easy Install / setuptools to manage package installation, and have read the "Custom Installation Locations" --> " "Traditional" PYTHONPATH-based Installation" section of the EasyInstall manual (among other sections :). It's key that, if easy_install installs prereq's, those packages end up installed in ~/pylib (since I don't have write access to any site-packages directories). The instructions at http://peak.telecommunity.com/DevCenter/EasyInstall indicate that I should probably have a ~/.pydistutils.cfg file containing either:: [easy_install] install_dir = /home/me/pylib or else:: [install] install_lib = /home/me/pylib install_scripts = /home/me/bin So, which one of those would be suitable for my setup? It looks like an alternative to that would be using the ``--install-dir`` option to easy_install every time I run it. If using ``--install-dir``, will easy_install honor that when decending to instal dependencies as well? Note, I've already got ~/py/bin in my PATH, PYTHONPATH is being set to /home/me/pylib in my ~/.bash_profile, and it looks to me that, installing packages by-hand, the correct way for my setup is:: python setup.py install --home=/home/me/pylib So, I'm guessing that the above configurations for easy_install will get me the same thing. Any tips appreciated. Thanks. ---John From strawman at astraw.com Tue May 15 08:29:49 2007 From: strawman at astraw.com (Andrew Straw) Date: Mon, 14 May 2007 23:29:49 -0700 Subject: [Distutils] stdeb bug: syntax error during dpkg -i In-Reply-To: <464849D7.2040702@pvv.org> References: <464232C3.7030300@pvv.org> <4643A59C.3010307@astraw.com> <46443FDF.2010104@astraw.com> <464849D7.2040702@pvv.org> Message-ID: <4649535D.3080903@astraw.com> Tore Ferner wrote: > Andrew Straw wrote: > > [snip] > > >> Hmm, this is strange. Apparently pycentral isn't interested in >> Python2.3, and yet it does get called. It's possible there's a pycentral >> bug, but I'm not sure. Can you post the contents of your >> deb-dst/bzr-0.16.0/debian/rules file? >> > > Hi, > > It's at the end of the mail. > Hmm, the line "PYVERS=$(shell pyversions -vr)" looks like the only place where Python 2.3 could get found. So, what does the output of "pyversions -vr" give you? Other that, I can't see anything else. Can you also post the output of "apt-cache show python-central" and "apt-cache show debhelper"? Thanks, Andrew From torfer at pvv.org Tue May 15 09:56:43 2007 From: torfer at pvv.org (Tore Ferner) Date: Tue, 15 May 2007 09:56:43 +0200 Subject: [Distutils] stdeb bug: syntax error during dpkg -i In-Reply-To: <4649535D.3080903@astraw.com> References: <464232C3.7030300@pvv.org> <4643A59C.3010307@astraw.com> <46443FDF.2010104@astraw.com> <464849D7.2040702@pvv.org> <4649535D.3080903@astraw.com> Message-ID: <464967BB.7040509@pvv.org> Andrew Straw wrote: > Tore Ferner wrote: >> Andrew Straw wrote: >> >> [snip] >> >> >>> Hmm, this is strange. Apparently pycentral isn't interested in >>> Python2.3, and yet it does get called. It's possible there's a pycentral >>> bug, but I'm not sure. Can you post the contents of your >>> deb-dst/bzr-0.16.0/debian/rules file? >>> >> >> Hi, >> >> It's at the end of the mail. >> > > Hmm, the line "PYVERS=$(shell pyversions -vr)" looks like the only place > where Python 2.3 could get found. So, what does the output of > "pyversions -vr" give you? When I stand in the location deb_dist/bzr-0.16.0, I get: $ pyversions -vr 2.4 I've found some reports about this "using old verion" "bug", but no solution really...yet. Maybe I just as well remove python2.3 completely... > Other that, I can't see anything else. Can > you also post the output of "apt-cache show python-central" and > "apt-cache show debhelper"? $ apt-cache show python-central Package: python-central Priority: standard Section: python Installed-Size: 200 Maintainer: Matthias Klose Architecture: all Version: 0.5.13-0.1 Depends: python (>= 2.3.5-7) Conflicts: debhelper (<= 5.0.37.3) Filename: pool/main/p/python-central/python-central_0.5.13-0.1_all.deb Size: 32360 MD5sum: 7755414e18725f10a175d55a86e3a5df SHA1: 94e410edcb6f3fea3fbd3fc7d888e0c199e1f0d0 SHA256: 5f08b32fb8ad086973e534fed730dfec005b575e890a33fa644e777f70ca7cfc Description: register and build utility for Python packages This package provides support for building and installing python modules independent of the current installed Python version. Tag: admin::automation, admin::package-management, devel::buildtools, devel::lang:python, implemented-in::python, interface::commandline, scope::utility, works-with::software:package, works-with::software:source ================================================ $ apt-cache show debhelper Package: debhelper Priority: optional Section: devel Installed-Size: 1260 Maintainer: Joey Hess Architecture: all Version: 5.0.49 Depends: perl (>= 5.6.0-16), coreutils, file (>= 3.23-1), dpkg-dev (>= 1.13.13), html2text, binutils, po-debconf Suggests: dh-make Conflicts: dpkg-cross (<< 1.18), python-support (<< 0.5.3), python-central (<< 0.5.6) Filename: pool/main/d/debhelper/debhelper_5.0.49_all.deb Size: 516948 MD5sum: b3af544027ede19ee72dc2f2f76e09b3 SHA1: f3d2899cd6831ac041c400b18a214fa44e6cf58f SHA256: c1c0484f491060610704575015d38821ef7ffb4b0856180d3f7be73376a7b0a7 Description: helper programs for debian/rules A collection of programs that can be used in a debian/rules file to automate common tasks related to building debian packages. Programs are included to install various files into your package, compress files, fix file permissions, integrate your package with the debian menu system, debconf, doc-base, etc. Most debian packages use debhelper as part of their build process. Tag: devel::{buildtools,debian,packaging}, implemented-in::perl, interface::commandline, role::program, scope::utility, suite::debian, works-with::software:source Package: debhelper Status: install ok installed Priority: optional Section: devel Installed-Size: 1256 Maintainer: Joey Hess Architecture: all Version: 5.0.44 Depends: perl (>= 5.6.0-16), coreutils, file (>= 3.23-1), dpkg-dev (>= 1.13.13), html2text, binutils, po-debconf Suggests: dh-make Conflicts: dpkg-cross (<< 1.18), python-support (<< 0.5.3), python-central (<< 0.5.6) Description: helper programs for debian/rules A collection of programs that can be used in a debian/rules file to automate common tasks related to building debian packages. Programs are included to install various files into your package, compress files, fix file permissions, integrate your package with the debian menu system, debconf, doc-base, etc. Most debian packages use debhelper as part of their build process. Package: debhelper Priority: optional Section: devel Installed-Size: 1004 Maintainer: Joey Hess Architecture: all Version: 4.2.32 Depends: perl (>= 5.6.0-16), coreutils | fileutils (>= 4.0-2.1), file (>= 3.23-1), dpkg-dev (>= 1.7.0), html2text, debconf-utils (>= 1.1.1), binutils, po-debconf Suggests: dh-make Conflicts: dpkg-cross (<< 1.18) Filename: pool/main/d/debhelper/debhelper_4.2.32_all.deb Size: 362154 MD5sum: 863cd01cfbbafda7946bb35df5bac4cf SHA1: 6a4e7e3389031d4c1dfbaf5373ad268382569eba SHA256: a6bdca9e8ec6c8acac7531dc2366e465caf88cfbca4ed074b7e79c2aa97ac97f Description: helper programs for debian/rules A collection of programs that can be used in a debian/rules file to automate common tasks related to building debian packages. Programs are included to install various files into your package, compress files, fix file permissions, integrate your package with the debian menu system, debconf, doc-base, etc. Most debian packages use debhelper as part of their build process. ================================================ $ dpkg -l| grep python-central ii python-central 0.5.13-0.1 register and build utility for Python packages $ dpkg -l| grep debhelper ii debhelper 5.0.44 helper programs for debian/rules Best regards Tore From hjebbers at gmail.com Tue May 15 12:38:05 2007 From: hjebbers at gmail.com (henk-jan ebbers) Date: Tue, 15 May 2007 12:38:05 +0200 Subject: [Distutils] import problem using setuptools Message-ID: <46498D8D.6030701@gmail.com> An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20070515/d0ca994a/attachment.htm From hjebbers at gmail.com Tue May 15 13:31:51 2007 From: hjebbers at gmail.com (henk-jan ebbers) Date: Tue, 15 May 2007 13:31:51 +0200 Subject: [Distutils] import problem using setuptools (resend without html) Message-ID: <46499A27.3030703@gmail.com> I am using setuptools to creat a windows installer (using python setup.py bdist --format=wininst). setuptools creates a 'entry point', so it generates a .exe in the Scripts directory (this exe calls start.py) /base/ start.py b.py /lib/ __init__.py c.py start.py imports b.py (which has a global that is initialised) then start.py imports c.py c.py imports start.py (as far as I know this is not a circular import, because I am using only 'import xx', not 'from xx import ..') when running start.py directly (as 'python start.py') everything goes OK. My problem is when I run the .exe from the entry point in the Scripts directory. when start.py is imported from c.py and start.py on its turn imports b.py, b.py is completly initialised again. This behavious does not happen when start.py is called directly. what am I doing wrong? I also noticed that, when creating and installing a windwos instaler, the easy-install.pth does not get an entry for my application. Is this intended? best regards, Henk-Jan From henk-jan at ebbersconsult.nl Tue May 15 13:23:27 2007 From: henk-jan at ebbersconsult.nl (henk-jan at ebbersconsult.nl) Date: Tue, 15 May 2007 13:23:27 +0200 Subject: [Distutils] import problem using setuptools (resend without html) Message-ID: <4649982F.2020709@ebbersconsult.nl> I am using setuptools to creat a windows installer (using python setup.py bdist --format=wininst). setuptools creates a 'entry point', so it generates a .exe in the Scripts directory (this exe calls start.py) /base/ start.py b.py /lib/ __init__.py c.py start.py imports b.py (which has a global that is initialised) then start.py imports c.py c.py imports start.py (as far as I know this is not a circular import, because I am using only 'import xx', not 'from xx import ..') when running start.py directly (as 'python start.py') everything goes OK. My problem is when I run the .exe from the entry point in the Scripts directory. when start.py is imported from c.py and start.py on its turn imports b.py, b.py is completly initialised again. This behavious does not happen when start.py is called directly. what am I doing wrong? I also noticed that, when creating and installing a windwos instaler, the easy-install.pth does not get an entry for my application. Is this intended? best regards, Henk-Jan From hjebbers at gmail.com Tue May 15 14:52:08 2007 From: hjebbers at gmail.com (henk-jan ebbers) Date: Tue, 15 May 2007 14:52:08 +0200 Subject: [Distutils] import problem using setuptools (resend without html) Message-ID: <4649ACF8.8020701@gmail.com> I examined which modules python has imported (sys.modules). when using the .exe in Scripts dir, most imports have 'mypackage' insert before module name, eg: mypackage.b but when importing from c.py, this does not happen; it is imported as b So python thinks it is another module.... This does not happen when running start.py directly (as 'python start.py') So I guess this is something done by setuptools, probably pkg_resources But...how do I solve this problem? best regards, Henk-Jan From pje at telecommunity.com Tue May 15 17:15:15 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 15 May 2007 11:15:15 -0400 Subject: [Distutils] import problem using setuptools (resend without html) In-Reply-To: <46499A27.3030703@gmail.com> References: <46499A27.3030703@gmail.com> Message-ID: <20070515151327.3E2073A4036@sparrow.telecommunity.com> At 01:31 PM 5/15/2007 +0200, henk-jan ebbers wrote: >I am using setuptools to creat a windows installer (using python >setup.py bdist --format=wininst). >setuptools creates a 'entry point', so it generates a .exe in the >Scripts directory (this exe calls start.py) > >/base/ > start.py > b.py > /lib/ > __init__.py > c.py > >start.py imports b.py (which has a global that is initialised) >then start.py imports c.py >c.py imports start.py (as far as I know this is not a circular import, >because I am using only 'import xx', not 'from xx import ..') > >when running start.py directly (as 'python start.py') everything goes OK. Are these the *actual* names of your files and directories? Please repost using actual real real names and paths, if not. If these *are* the actual names of the files and directories, the problem is that you are using an out-of-date version of setuptools that doesn't put "-script" in the filenames of scripts to be run using an .exe. That, and there is probably also something terribly wrong with your setup.py. However, if the above are *not* the real names of the files and directories involved, please repost and provide them, because it might be something else altogether. >I also noticed that, when creating and installing a windwos instaler, >the easy-install.pth does not get an entry for my application. Is this >intended? Yes. When you create "bdist_wininst", "bdist_msi", or "bdist_rpm" (or indeed most any "bdist" except bdist_egg), you should get a package layout identical to that produced by normal distutils, but with the addition of an .egg-info directory alongside the package. This layout doesn't require any .pth files. From hjebbers at gmail.com Tue May 15 17:37:06 2007 From: hjebbers at gmail.com (henk-jan ebbers) Date: Tue, 15 May 2007 17:37:06 +0200 Subject: [Distutils] import problem using setuptools (resend without html) In-Reply-To: <20070515151327.3E2073A4036@sparrow.telecommunity.com> References: <46499A27.3030703@gmail.com> <20070515151327.3E2073A4036@sparrow.telecommunity.com> Message-ID: <4649D3A2.8070905@gmail.com> An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20070515/d149a4cb/attachment.html From pje at telecommunity.com Tue May 15 18:29:56 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 15 May 2007 12:29:56 -0400 Subject: [Distutils] import problem using setuptools (resend without html) In-Reply-To: <4649D3A2.8070905@gmail.com> References: <46499A27.3030703@gmail.com> <20070515151327.3E2073A4036@sparrow.telecommunity.com> <4649D3A2.8070905@gmail.com> Message-ID: <20070515162808.5E5413A4036@sparrow.telecommunity.com> At 05:37 PM 5/15/2007 +0200, henk-jan ebbers wrote: >Thanks for your response. >Yes, these are not the real names. >My situation is (using real names, but using only relevant files): > >/bots/ > engine.py > transform.py > botslib.py > __init__.py > /usersys/ > __init__.py > /mappings/ > __init__.py > 02ordersedi2aperak.py > >so: >engine.py is the start point. >engine.py imports botslib.py (initialises global), imports >transform.py and starts function from transform.py >transform imports botslib.py, imports usersys.mappings.02ordersedi2aperak.py >usersys.mappings.02ordersedi2aperak.py imports botslib.py->in this >import it is initialised again. Ah, then your code has a bug. It should be importing bots.botslib, instead (e.g. "from bots import botslib" instead of "import botslib"). It appears that you also either have the bots/ directory somehow appearing on sys.path (where it *should not be*!) OR you have another copy of botslib.py on sys.path. From pje at telecommunity.com Tue May 15 19:48:40 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 15 May 2007 13:48:40 -0400 Subject: [Distutils] Overly picky about file URLs In-Reply-To: <80498EC4-E2E5-42E2-BA3F-A25CBB7B4200@zope.com> References: <80498EC4-E2E5-42E2-BA3F-A25CBB7B4200@zope.com> Message-ID: <20070515174651.7E2113A4036@sparrow.telecommunity.com> At 10:34 AM 5/12/2007 -0400, Jim Fulton wrote: >The local_open function in setuptools.package_index is used to open >file URLs. A file URL pointing to a directory is treated as non >found unless it ends in a "/". This seems overly restrictive to me. >Would anyone object if the restriction was relaxed? That restriction is so that the generated relative URLs work correctly. In order to relax that restriction, you'll need to change the generated URLs in the index, if a trailing / wasn't used. I just punted on that case to keep it simple, but feel free to patch it if you like (trunk and branch, of course). From pje at telecommunity.com Tue May 15 19:50:16 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 15 May 2007 13:50:16 -0400 Subject: [Distutils] setuptools process_url doesn't handle URLErrors gracefullly In-Reply-To: References: <1178871702.22778.8.camel@mindy.whq.gocept.com> Message-ID: <20070515174827.ECD2E3A4036@sparrow.telecommunity.com> At 09:52 AM 5/14/2007 -0400, Jim Fulton wrote: >This problem is rather serious. As we rely more and more on >setuptools, it needs to be robust in the presence of this sort of >failure. > >Phillip, If I fix this, can we get a timely release with the fix? Is sometime next week sufficient to be described as timely? If so, then yes. From pje at telecommunity.com Tue May 15 19:59:57 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 15 May 2007 13:59:57 -0400 Subject: [Distutils] automating __version__' ing issues In-Reply-To: References: <20070514003734.7EF9C3A409B@sparrow.telecommunity.com> Message-ID: <20070515175808.16E963A40A9@sparrow.telecommunity.com> At 01:58 PM 5/14/2007 +0100, Alexander Schmolck wrote: >"Phillip J. Eby" writes: > > At 01:09 AM 5/14/2007 +0100, Alexander Schmolck wrote: > >>"Phillip J. Eby" writes: > >> > >> >>Here's the rub-- it must be in setup.py and you probably want > >> >>to give your package access to it so that it can be reported. It > >> >>appears that the current idiom for solving this dilemma is to but a > >> >>release.py or version.py file in your package, which is sucked up into > >> >>the setup.py file with execfile (see > >> >> for an > >> >>example). You then manually maintain the version number in one place > >> >>(the release.py file). > >> > > >> > Yep, this is what most people do. > >> > >>Is there a standard trick to get the desired svn-revision? Running > >>`'svnversion > __svn_version__.py`` in setup.py when a distrubtion-building > >>command is issued and then importing that in a hand-maintained > >>``version.py``/``release.py`` file should work, but is there a way to check > >>whether a setup.py command falls into a certain category? > > > > I don't understand what you're trying to do. > >I'd like to have a mechanism that has me specify two things in *one* central >location: > >1. A self-chosen version number, e.g. "1.1" Put this in setup.py. >2. A release flag (e.g. official_release=False, meaning it's an official and > not a development release) Put this in setup.cfg for a development release (or leave blank for official): [egg_info] tag_svn_revision = 1 tag_build = dev >Now in all other locations I'd like to get a version string that either is >"1.1" if release is True "1.1.dev43" (or whatever the svn revision) if release >is False. > >All other locations means at least setup.py and a __init__.py (via import or >execfile if needs be). At runtime, you can use this code to get the version of your project (or any other project): from pkg_resources import require __version__ = require('MyProjectname')[0].version Where 'MyProjectname' is your setup(name=...) name. (I.e., the name of your distribution.) You can also do this at the interpreter prompt, e.g.: Python 2.3.4 (#53, May 25 2004, 21:17:02) [MSC v.1200 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> from pkg_resources import require >>> require('PEAK') [peak 0.5a4.dev-r2287 (c:\python23\lib\site-packages\peak-0.5a4.dev_r2287-py2.3-win32.egg), DecoratorTools 1.4 (c:\cygwin\home\pje\projects\decoratortools), zconfig 2.3 (c:\python23\lib\site-packages\zconfig-2.3-py2.3.egg), wsgiref 0.1.2 (c:\cygwin\home\pje\projects\wsgiref), SymbolType 1.0 (c:\cygwin\home\pje\projects\symboltype), Importing 1.9.3 (c:\cygwin\home\pje\projects\importing), PyProtocols 1.0a0dev-r2302 (c:\cygwin\home\pje\projects\pyprotocols\src), ruledispatch 0.5a0.dev-r2287 (c:\python23\lib\site-packages\ruledispatch-0.5a0.dev_r2287-py2.3-win32.egg)] >>> As you can see, require() returns a list of objects representing all the packages needed to meet the given request, and the default repr() inlcudes both version numbers and file paths. This is really the best way to find out what version(s) of something somebody is using, as it includes your dependencies as well as the base package. >p.s. I notice that a cc: or to: to your email address bounces -- is that >intended? No, and AFAIK it's not happening to anybody else. From a.schmolck at gmx.net Tue May 15 22:14:43 2007 From: a.schmolck at gmx.net (Alexander Schmolck) Date: Tue, 15 May 2007 21:14:43 +0100 Subject: [Distutils] automating __version__' ing issues In-Reply-To: <20070515175808.16E963A40A9@sparrow.telecommunity.com> (Phillip J. Eby's message of "Tue\, 15 May 2007 13\:59\:57 -0400") References: <20070515175808.16E963A40A9@sparrow.telecommunity.com> Message-ID: "Phillip J. Eby" writes: >>I'd like to have a mechanism that has me specify two things in *one* central >>location: >> >>1. A self-chosen version number, e.g. "1.1" > > Put this in setup.py. > > >>2. A release flag (e.g. official_release=False, meaning it's an official and >> not a development release) > > Put this in setup.cfg for a development release (or leave blank for official): > > [egg_info] > tag_svn_revision = 1 > tag_build = dev Will do. But wouldn't it make sense to be able to specify the version name in this place to? >>Now in all other locations I'd like to get a version string that either is >>"1.1" if release is True "1.1.dev43" (or whatever the svn revision) if release >>is False. >> >>All other locations means at least setup.py and a __init__.py (via import or >>execfile if needs be). > > At runtime, you can use this code to get the version of your project (or any > other project): > > from pkg_resources import require > __version__ = require('MyProjectname')[0].version > > Where 'MyProjectname' is your setup(name=...) name. (I.e., the name of your > distribution.) Just to make sure because I'm not 100% clear on this and I think I have have come across conflicting information from different sources -- if my package is named scikits.mlabwrap and scikits is a namespace package, setup.py will have to look like so, right? setup(name='sckits.mlabwrap', # rather than just 'mlabwrap' namespace_packages=['scikits'] ... ) and then ``require(scikits.mlabwrap)[0].version`` will get me the version number. As a suggestion: I think that the ``namespace_packages`` example for the ZoperInterface project at would profit from adding the ``name=`` parameter, because it is not immediately obvious whether it should include the namespace_package or not. > [...] Thanks for all the info! I've tried to document the most important points about setuptools for other scikits developers, if you or someone else who's more knowledgable about setuptools wants to give some feedback (although it's obviously still work in progress), here's the url: Thanks again for your help, cheers 'as >>p.s. I notice that a cc: or to: to your email address bounces -- is that >>intended? > > No, and AFAIK it's not happening to anybody else. FWIW, here's the gist of the error message: A message that you sent could not be delivered to one or more of its recipients. This is a permanent error. The following address(es) failed: pje at telecommunity.com SMTP error from remote mail server after RCPT TO:: host mail.telecommunity.com [209.190.5.234]: 550 : Recipient address rejected: SPF Reports: SPF fail - not authorized ------ This is a copy of the message, including all the headers. ------ Return-path: Received: from oc.ex.ac.uk ([144.173.207.34]) by dot.ex.ac.uk with esmtp (Exim 4.62/mail) id 1HnLB1-0003Br-Sq; Sun, 13 May 2007 22:00:23 +0100 To: "Phillip J. Eby" Cc: distutils-sig at python.org, brian.lee.hawthorne at gmail.com Subject: Re: [Distutils] unpacking files from and egg only for testing References: <20070513162117.C061E3A40B0 at sparrow.telecommunity.com> From: Alexander Schmolck Date: Sun, 13 May 2007 22:00:23 +0100 In-Reply-To: <20070513162117.C061E3A40B0 at sparrow.telecommunity.com> (Phillip J. Eby's message of "Sun\, 13 May 2007 12\:23\:04 -0400") Message-ID: User-Agent: Gnus/5.11 (Gnus v5.11) Emacs/23.0.0 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii b -- From hjebbers at gmail.com Tue May 15 22:27:29 2007 From: hjebbers at gmail.com (henk-jan ebbers) Date: Tue, 15 May 2007 22:27:29 +0200 Subject: [Distutils] import problem using setuptools (resend without html) In-Reply-To: <20070515162808.5E5413A4036@sparrow.telecommunity.com> References: <46499A27.3030703@gmail.com> <20070515151327.3E2073A4036@sparrow.telecommunity.com> <4649D3A2.8070905@gmail.com> <20070515162808.5E5413A4036@sparrow.telecommunity.com> Message-ID: <464A17B1.30804@gmail.com> An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20070515/388eb090/attachment.html From pje at telecommunity.com Tue May 15 22:46:12 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 15 May 2007 16:46:12 -0400 Subject: [Distutils] automating __version__' ing issues In-Reply-To: References: <20070515175808.16E963A40A9@sparrow.telecommunity.com> Message-ID: <20070515204424.6C4963A4036@sparrow.telecommunity.com> At 09:14 PM 5/15/2007 +0100, Alexander Schmolck wrote: >"Phillip J. Eby" writes: >Will do. But wouldn't it make sense to be able to specify the version name in >this place to? Well, the version is a distribution level option; it can't be specified in setup.cfg. That's a distutils thing, I'm afraid. Btw, I erred in one point in my last email, your setup.cfg should actually read: [egg_info] tag_svn_revision = 1 tag_build = .dev Note the extra '.' - this ensures that if you have something like '1.1a' as your version, you won't end up with '1.1adev' as your version. > > Where 'MyProjectname' is your setup(name=...) name. (I.e., the > name of your > > distribution.) > >Just to make sure because I'm not 100% clear on this and I think I have have >come across conflicting information from different sources -- if my package is >named scikits.mlabwrap and scikits is a namespace package, setup.py will have >to look like so, right? > > setup(name='sckits.mlabwrap', # rather than just 'mlabwrap' > namespace_packages=['scikits'] > ... > ) Uh, no. Your distribution name could be "Matlab Wrapper for SciKits" or "FooBarBaz", for all the distutils or setuptools care. :) Project names don't have any defined relationship to package names. For example the "Importing" project on the Cheeseshop provides a module called "peak.util.imports". >and then ``require(scikits.mlabwrap)[0].version`` will get me the version >number. You need quotes around the scikits.mlabwrap... or 'FooBarBaz', if that's your project name. :) >As a suggestion: I think that the ``namespace_packages`` example for the >ZoperInterface project at > would >profit from adding the ``name=`` parameter, because it is not immediately >obvious whether it should include the namespace_package or not. It's entirely unrelated - note that a project can contain multiple packages, or even no packages at all. >Thanks for all the info! I've tried to document the most important points >about setuptools for other scikits developers, if you or someone else who's >more knowledgable about setuptools wants to give some feedback (although it's >obviously still work in progress), here's the url: > > Reading that page, I'm not sure whether you want a scikits namespace package in the first place. A namespace package is used to group several projects under a single "owner", ala the various zope.* and peak.* and plone.* packages. However, nobody but Zope Corp. should be releasing a zope.* package, and nobody but me releasing a peak.* one. The whole point is to "claim" a namespace for packages being released by a single owner - either an organization or umbrella project. This corresponds to things like the 'org.apache' and 'com.sun' package namespaces in Java - it's just a namespace to allow authors free rein to put whatever modules and packages they like under that namespace. However, it sounds from that page like scikits are independently created - which means there's no need for a namespace package there, and in fact it's counter-indicated for that situation! Also, may I suggest that rather than having a __version__ variable, why not just have a get_version() function? That way, you needn't import pkg_resources or do a require() as part of a simple import operation. Finally, it's not the case that you mustn't write a MANIFEST.in; setuptools will use MANIFEST.in just fine. What setuptools *won't* do is rely on a generated MANIFEST for anything; it always uses a freshly-generated internal manifest, using MANIFEST.in + revision control info. Thus, it's impossible to mess up setuptools with a stale MANIFEST file, the way you can with distutils. (And if you're using CVS or SVN, you mostly won't need to bother with MANIFEST.in.) From pje at telecommunity.com Tue May 15 23:20:37 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 15 May 2007 17:20:37 -0400 Subject: [Distutils] import problem using setuptools (resend without html) In-Reply-To: <464A17B1.30804@gmail.com> References: <46499A27.3030703@gmail.com> <20070515151327.3E2073A4036@sparrow.telecommunity.com> <4649D3A2.8070905@gmail.com> <20070515162808.5E5413A4036@sparrow.telecommunity.com> <464A17B1.30804@gmail.com> Message-ID: <20070515211848.E3D9A3A4036@sparrow.telecommunity.com> At 10:27 PM 5/15/2007 +0200, henk-jan ebbers wrote: >You are right on both points. >thank you. this works. >I am a bit surprised that I should use 'import transform.py' in the >/bots/ directory, You should not. In Python 2.6 and 3.0 it will not work any more. Use "import bots.transform" or "from bots import transform" instead. >and 'import bots.transform' in the /mapping/ directory. Under Python 2.5, you also have the option of doing 'from .. import transform' to import the transform module from the package above you, or 'from . import transform' to import it from the package you are in. See PEP 328 at http://www.python.org/dev/peps/pep-0328/ for more info. From robert.kern at gmail.com Tue May 15 23:47:21 2007 From: robert.kern at gmail.com (Robert Kern) Date: Tue, 15 May 2007 16:47:21 -0500 Subject: [Distutils] automating __version__' ing issues In-Reply-To: <20070515204424.6C4963A4036@sparrow.telecommunity.com> References: <20070515175808.16E963A40A9@sparrow.telecommunity.com> <20070515204424.6C4963A4036@sparrow.telecommunity.com> Message-ID: Phillip J. Eby wrote: > Reading that page, I'm not sure whether you want a scikits namespace > package in the first place. A namespace package is used to group > several projects under a single "owner", ala the various zope.* and > peak.* and plone.* packages. However, nobody but Zope Corp. should > be releasing a zope.* package, and nobody but me releasing a peak.* > one. The whole point is to "claim" a namespace for packages being > released by a single owner - either an organization or umbrella > project. This corresponds to things like the 'org.apache' and > 'com.sun' package namespaces in Java - it's just a namespace to allow > authors free rein to put whatever modules and packages they like > under that namespace. This is exactly what we are aiming for. While different people will be responsbile for each subpackage, they are all operating under the aegis of the scikits infrastructure. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From pje at telecommunity.com Tue May 15 23:51:09 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 15 May 2007 17:51:09 -0400 Subject: [Distutils] SPF error (was Re: automating __version__' ing issues) In-Reply-To: References: <20070515175808.16E963A40A9@sparrow.telecommunity.com> Message-ID: <20070515214921.E1EEF3A4036@sparrow.telecommunity.com> Btw, I forgot to address this part earlier... At 09:14 PM 5/15/2007 +0100, Alexander Schmolck wrote: > >>p.s. I notice that a cc: or to: to your email address bounces -- is that > >>intended? > > > > No, and AFAIK it's not happening to anybody else. > >FWIW, here's the gist of the error message: > >A message that you sent could not be delivered to one or more of its >recipients. This is a permanent error. The following address(es) failed: > > pje at telecommunity.com > SMTP error from remote mail server after RCPT TO:: > host mail.telecommunity.com [209.190.5.234]: 550 : > Recipient address rejected: SPF Reports: SPF fail - not authorized > >------ This is a copy of the message, including all the headers. ------ > >Return-path: >Received: from oc.ex.ac.uk ([144.173.207.34]) > by dot.ex.ac.uk with esmtp (Exim 4.62/mail) > id 1HnLB1-0003Br-Sq; Sun, 13 May 2007 22:00:23 +0100 You're sending your message with a return address @gmx.net, but gmx.net's SPF records read: gmx.net. 300 IN TXT "v=spf1 ip4:213.165.64.0/23 -all" Which means that your ISP says that anybody using a return address @gmx.net who isn't sending their mail via the 213.165.64.0/23 IP block is a lying spammer pretending to have a gmx.net address. :) Thus, any SPF-supporting mail server is going to think you're a spammer and block your mail, unless you send it through a gmx.net-supported SMTP server. From a.schmolck at gmx.net Wed May 16 00:27:50 2007 From: a.schmolck at gmx.net (Alexander Schmolck) Date: Tue, 15 May 2007 23:27:50 +0100 Subject: [Distutils] automating __version__' ing issues In-Reply-To: <20070515204424.6C4963A4036@sparrow.telecommunity.com> (Phillip J. Eby's message of "Tue\, 15 May 2007 16\:46\:12 -0400") References: <20070515204424.6C4963A4036@sparrow.telecommunity.com> Message-ID: "Phillip J. Eby" writes: > At 09:14 PM 5/15/2007 +0100, Alexander Schmolck wrote: >>"Phillip J. Eby" writes: >>Will do. But wouldn't it make sense to be able to specify the version name in >>this place to? > > Well, the version is a distribution level option; it can't be specified in > setup.cfg. That's a distutils thing, I'm afraid. Why can't [egg_info] just grow a 'version' field? Due to the way the code is organized? I don't really understand why, logically, the .dev suffix is not distribution level -- it forms part of the .egg version (and what can be required/queried by other packages) after all... > > Btw, I erred in one point in my last email, your setup.cfg should actually > read: > > [egg_info] > tag_svn_revision = 1 > tag_build = .dev > > Note the extra '.' - this ensures that if you have something like '1.1a' as > your version, you won't end up with '1.1adev' as your version. Thanks, corrected > > >> > Where 'MyProjectname' is your setup(name=...) name. (I.e., the name of >> your >> > distribution.) >> >>Just to make sure because I'm not 100% clear on this and I think I have have >>come across conflicting information from different sources -- if my package is >>named scikits.mlabwrap and scikits is a namespace package, setup.py will have >>to look like so, right? >> >> setup(name='sckits.mlabwrap', # rather than just 'mlabwrap' >> namespace_packages=['scikits'] >> ... >> ) > > Uh, no. Your distribution name could be "Matlab Wrapper for SciKits" or > "FooBarBaz", for all the distutils or setuptools care. :) Project names > don't have any defined relationship to package names. For example the > "Importing" project on the Cheeseshop provides a module called > "peak.util.imports". OK, I see (I guess I must have been aware of that once). Maybe it would be good to state this somewhere were it's easy to find -- try searching for "name" on the setuptools page to see what I mean -- 'name' occurs in all sort of combinations and the first body text occurence is in , but I it's meaning is not further explained there. Maybe something like: "Note that the ``name = "HelloWorld"`` option has no other effect than specifiying what the distribtuion name of your package will be [[i.e. .egg name and cheeseshop?]]; in particular it is completely independent of the package name itself (i.e. the package import statement could in principle be ``import goodbye.cruel.world`` rather than ``import HelloWorld``)." or just name='HelloWorld' # what the project will be listed as in Cheeseshop >>and then ``require(scikits.mlabwrap)[0].version`` will get me the version >>number. > > You need quotes around the scikits.mlabwrap... or 'FooBarBaz', if that's your > project name. :) Yup, typo. >>As a suggestion: I think that the ``namespace_packages`` example for the >>ZoperInterface project at >> >>would profit from adding the ``name=`` parameter, because it is not >>immediately obvious whether it should include the namespace_package or not. > > It's entirely unrelated - note that a project can contain multiple packages, > or even no packages at all. Actually mlabwrap itself once had several toplevel modules -- in distutils/pre-scikits days -- so as I said, I likely was once aware of this :) >>Thanks for all the info! I've tried to document the most important points >>about setuptools for other scikits developers, if you or someone else who's >>more knowledgable about setuptools wants to give some feedback (although it's >>obviously still work in progress), here's the url: >> >> > > Reading that page, I'm not sure whether you want a scikits namespace package > in the first place. A namespace package is used to group several projects > under a single "owner", ala the various zope.* and peak.* and plone.* > packages. However, nobody but Zope Corp. should be releasing a zope.* > package, and nobody but me releasing a peak.* one. The whole point is to > "claim" a namespace for packages being released by a single owner - either > an organization or umbrella project. I think this holds true here (the umbrella community being scipy). Although the barriers for entry are lower than for scipy core, not anyone who feels like it can just create a scikits package -- for one thing someone needs to give you svn etc. access (somone = Jeff Strunk from enthought). > This corresponds to things like the org.apache' and 'com.sun' package > namespaces in Java - it's just a namespace to allow authors free rein to put > whatever modules and packages they like under that namespace. However, it > sounds from that page like scikits are independently created - which means > there's no need for a namespace package there, and in fact it's > counter-indicated for that situation! Well, independent as in not (necessarily) having mutual dependencies or requiring joint install (although pretty much all packages will need scipy or at least numpy) -- not as in having absolutely nothing to do with each other or being free for all. I think[1] an publication in a specilized journal run by a dedicated scientific community (say the ACM's APLers) might be vaguely analogous -- it provides a way for people who are interested in a particular niche to find pool and find relevant stuff that passed at least some weak form of peer-review to get to be associated with the ACM brand. > Also, may I suggest that rather than having a __version__ variable, why not > just have a get_version() function? The great advantage of __version__ is that it's standard. I personally frequently look up the version of an installed package by typing .__v and I'm seldomly disappointed; it wouldn't occur to me to look for a get_version function. > That way, you needn't import pkg_resources or do a require() as part of a > simple import operation. I hope it doesn't potentially take ages? For mlabwrap it wouldn't matter much (matlab takes ages to start up on import), but for other scikits this might be a problem. > Finally, it's not the case that you mustn't write a MANIFEST.in; setuptools > will use MANIFEST.in just fine. What setuptools *won't* do is rely on a > generated MANIFEST for anything; it always uses a freshly-generated internal > manifest, using MANIFEST.in + revision control info. Thus, it's impossible to > mess up setuptools with a stale MANIFEST file, the way you can with distutils. > (And if you're using CVS or SVN, you mostly won't need to bother with > MANIFEST.in.) OK, thanks for the clarification -- I've edited the webpage. cheers, 'as Footnotes: [1] The whole scikits stuff isn't my idea and its not fully fleshed out yet, so I'm just guessing wildly :) Robert Kern is more qualified to say something about the scope and aims of scikits. From jim at zope.com Wed May 16 00:33:35 2007 From: jim at zope.com (Jim Fulton) Date: Tue, 15 May 2007 18:33:35 -0400 Subject: [Distutils] Overly picky about file URLs In-Reply-To: <20070515174651.7E2113A4036@sparrow.telecommunity.com> References: <80498EC4-E2E5-42E2-BA3F-A25CBB7B4200@zope.com> <20070515174651.7E2113A4036@sparrow.telecommunity.com> Message-ID: <17846F4A-F132-407E-BEC2-6A593F1B25F5@zope.com> On May 15, 2007, at 1:48 PM, Phillip J. Eby wrote: > At 10:34 AM 5/12/2007 -0400, Jim Fulton wrote: >> The local_open function in setuptools.package_index is used to open >> file URLs. A file URL pointing to a directory is treated as non >> found unless it ends in a "/". This seems overly restrictive to me. >> Would anyone object if the restriction was relaxed? > > That restriction is so that the generated relative URLs work > correctly. In order to relax that restriction, you'll need to > change the generated URLs in the index, if a trailing / wasn't > used. I just punted on that case to keep it simple, but feel free > to patch it if you like (trunk and branch, of course). I don't understand this. In particular, I don't understand why rejecting file URLs that point to directories but don't have trailing slashes makes relative URLs easier. I propose to change: if os.path.isfile(filename): return urllib2.urlopen(url) elif path.endswith('/') and os.path.isdir(filename): in package_index.local_open to: if os.path.isfile(filename): return urllib2.urlopen(url) elif os.path.isdir(filename): I don't really understand how this would affect relative URLs. Jim -- Jim Fulton mailto:jim at zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From pje at telecommunity.com Wed May 16 01:11:16 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 15 May 2007 19:11:16 -0400 Subject: [Distutils] automating __version__' ing issues In-Reply-To: References: <20070515175808.16E963A40A9@sparrow.telecommunity.com> <20070515204424.6C4963A4036@sparrow.telecommunity.com> Message-ID: <20070515230928.E185B3A4036@sparrow.telecommunity.com> At 04:47 PM 5/15/2007 -0500, Robert Kern wrote: >Phillip J. Eby wrote: > > > Reading that page, I'm not sure whether you want a scikits namespace > > package in the first place. A namespace package is used to group > > several projects under a single "owner", ala the various zope.* and > > peak.* and plone.* packages. However, nobody but Zope Corp. should > > be releasing a zope.* package, and nobody but me releasing a peak.* > > one. The whole point is to "claim" a namespace for packages being > > released by a single owner - either an organization or umbrella > > project. This corresponds to things like the 'org.apache' and > > 'com.sun' package namespaces in Java - it's just a namespace to allow > > authors free rein to put whatever modules and packages they like > > under that namespace. > >This is exactly what we are aiming for. While different people will be >responsbile for each subpackage, they are all operating under the aegis of the >scikits infrastructure. But who adds things to the namespace? Is there a single organization that decides what packages or modules will exist in scikits? From pje at telecommunity.com Wed May 16 01:12:01 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 15 May 2007 19:12:01 -0400 Subject: [Distutils] Overly picky about file URLs In-Reply-To: <17846F4A-F132-407E-BEC2-6A593F1B25F5@zope.com> References: <80498EC4-E2E5-42E2-BA3F-A25CBB7B4200@zope.com> <20070515174651.7E2113A4036@sparrow.telecommunity.com> <17846F4A-F132-407E-BEC2-6A593F1B25F5@zope.com> Message-ID: <20070515231012.077633A4036@sparrow.telecommunity.com> At 06:33 PM 5/15/2007 -0400, Jim Fulton wrote: >On May 15, 2007, at 1:48 PM, Phillip J. Eby wrote: > >>At 10:34 AM 5/12/2007 -0400, Jim Fulton wrote: >>>The local_open function in setuptools.package_index is used to open >>>file URLs. A file URL pointing to a directory is treated as non >>>found unless it ends in a "/". This seems overly restrictive to me. >>>Would anyone object if the restriction was relaxed? >> >>That restriction is so that the generated relative URLs work >>correctly. In order to relax that restriction, you'll need to >>change the generated URLs in the index, if a trailing / wasn't >>used. I just punted on that case to keep it simple, but feel free >>to patch it if you like (trunk and branch, of course). > >I don't understand this. In particular, I don't understand why >rejecting file URLs that point to directories but don't have trailing >slashes makes relative URLs easier. > >I propose to change: > > if os.path.isfile(filename): > return urllib2.urlopen(url) > elif path.endswith('/') and os.path.isdir(filename): > >in package_index.local_open to: > > if os.path.isfile(filename): > return urllib2.urlopen(url) > elif os.path.isdir(filename): > >I don't really understand how this would affect relative URLs. The relative URLs that are in the HTML that local_open() returns will be broken if you do that. From peter.mowry at amd.com Wed May 16 01:11:36 2007 From: peter.mowry at amd.com (Mowry, Peter) Date: Tue, 15 May 2007 18:11:36 -0500 Subject: [Distutils] setup.py demo.C build (gcc vs. g++) (distutils) In-Reply-To: <4647DD2F.3000706@matt-good.net> References: Your message of "Sat, 12 May 2007 19:09:55 EST." <475F4CA05E4E1B46A7213674E70EFA62022968CA@SAUSEXMB2.amd.com> <15086.1179025179@d.0x1.org> <475F4CA05E4E1B46A7213674E70EFA6202296938@SAUSEXMB2.amd.com> <4647DD2F.3000706@matt-good.net> Message-ID: <475F4CA05E4E1B46A7213674E70EFA620237D420@SAUSEXMB2.amd.com> Ever seen this error? Build Error Description: python2.5 setup.py demo.C build That creates a demo.so, and when I try to import it, it gives me the error: ImportError: ./demo.so: undefined symbol: __gxx_personality_v0 Hack to fix it: I found one single helpful post about this on google: http://www.intevation.de/pipermail/thuban-list/2003-September/000230.htm l suggests: Do the link step of gdalwarp manually by copying the commandline printed by setup.py and replacing the gcc with g++ So I tried this, and I saw that setup.py prints the following 2 commands: gcc -pthread -fno-strict-aliasing -DNDEBUG -g -O3 -Wstrict-prototypes -fpermissive -fPIC -I/tool/pandora64/.package/python-2.5/include/python2.5 -c demo.C -o build/temp.linux-x86_64-2.5/demo.o gcc -pthread -shared build/temp.linux-x86_64-2.5/demo.o -o build/lib.linux-x86_64-2.5/demo.so Then I simply manually rerun just the second command with g++: g++ -pthread -shared build/temp.linux-x86_64-2.5/demo.o -o build/lib.linux-x86_64-2.5/demo.so And now the import works and so does the python spam_system() function. Non-Hack to fix it?: Is there a non-hack way to fix this? I've seen this question on the internet but no answers so far. I did check --help-compiler, and it only lists --compiler=unix, not --compiler=g++ for ex. --Thanks Peter Mowry ("Pem") SimNow Team, Software Engineer AMD Organization -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20070515/63254319/attachment.htm -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/gif Size: 73 bytes Desc: image001.gif Url : http://mail.python.org/pipermail/distutils-sig/attachments/20070515/63254319/attachment.gif From robert.kern at gmail.com Wed May 16 01:19:53 2007 From: robert.kern at gmail.com (Robert Kern) Date: Tue, 15 May 2007 18:19:53 -0500 Subject: [Distutils] automating __version__' ing issues In-Reply-To: <20070515230928.E185B3A4036@sparrow.telecommunity.com> References: <20070515175808.16E963A40A9@sparrow.telecommunity.com> <20070515204424.6C4963A4036@sparrow.telecommunity.com> <20070515230928.E185B3A4036@sparrow.telecommunity.com> Message-ID: Phillip J. Eby wrote: > At 04:47 PM 5/15/2007 -0500, Robert Kern wrote: >> Phillip J. Eby wrote: >> >>> Reading that page, I'm not sure whether you want a scikits namespace >>> package in the first place. A namespace package is used to group >>> several projects under a single "owner", ala the various zope.* and >>> peak.* and plone.* packages. However, nobody but Zope Corp. should >>> be releasing a zope.* package, and nobody but me releasing a peak.* >>> one. The whole point is to "claim" a namespace for packages being >>> released by a single owner - either an organization or umbrella >>> project. This corresponds to things like the 'org.apache' and >>> 'com.sun' package namespaces in Java - it's just a namespace to allow >>> authors free rein to put whatever modules and packages they like >>> under that namespace. >> This is exactly what we are aiming for. While different people will be >> responsbile for each subpackage, they are all operating under the aegis of the >> scikits infrastructure. > > But who adds things to the namespace? Is there a single organization > that decides what packages or modules will exist in scikits? Yes, the scipy developer team. mlabwrap is just the first scikits project. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From pje at telecommunity.com Wed May 16 02:11:06 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 15 May 2007 20:11:06 -0400 Subject: [Distutils] setup.py demo.C build (gcc vs. g++) (distutils) In-Reply-To: <475F4CA05E4E1B46A7213674E70EFA620237D420@SAUSEXMB2.amd.com > References: <15086.1179025179@d.0x1.org> <475F4CA05E4E1B46A7213674E70EFA6202296938@SAUSEXMB2.amd.com> <4647DD2F.3000706@matt-good.net> <475F4CA05E4E1B46A7213674E70EFA620237D420@SAUSEXMB2.amd.com> Message-ID: <20070516000919.279273A4036@sparrow.telecommunity.com> At 06:11 PM 5/15/2007 -0500, Mowry, Peter wrote: >Content-class: urn:content-classes:message >Content-Type: multipart/related; > boundary="----_=_NextPart_001_01C79746.62B92D4C"; > type="multipart/alternative" > >Ever seen this error? > >Build Error Description: >python2.5 setup.py demo.C build > >That creates a demo.so, and when I try to import it, it gives me the error: >ImportError: ./demo.so: undefined symbol: __gxx_personality_v0 > > >Hack to fix it: >I found one single helpful post about this on google: >http://www.intevation.de/pipermail/thuban-list/2003-September/000230.html >suggests: > >Do the link step of gdalwarp manually by copying the commandline >printed by setup.py and replacing the gcc with g++ >So I tried this, and I saw that setup.py prints the following 2 commands: >gcc -pthread -fno-strict-aliasing -DNDEBUG -g -O3 >-Wstrict-prototypes -fpermissive -fPIC >-I/tool/pandora64/.package/python-2.5/include/python2.5 -c demo.C -o >build/temp.linux-x86_64-2.5/demo.o >gcc -pthread -shared build/temp.linux-x86_64-2.5/demo.o -o >build/lib.linux-x86_64-2.5/demo.so > >Then I simply manually rerun just the second command with g++: >g++ -pthread -shared build/temp.linux-x86_64-2.5/demo.o -o >build/lib.linux-x86_64-2.5/demo.so > >And now the import works and so does the python spam_system() function. > > >Non-Hack to fix it?: >Is there a non-hack way to fix this? > >I've seen this question on the internet but no answers so far. I >did check --help-compiler, and it only lists --compiler=unix, not >--compiler=g++ for ex. I believe you can use the $CC environment variable, e.g. CC=g++ python setup.py whatever should cause it to use g++ as the executable instead of gcc. From peter.mowry at amd.com Wed May 16 04:56:53 2007 From: peter.mowry at amd.com (Mowry, Peter) Date: Tue, 15 May 2007 21:56:53 -0500 Subject: [Distutils] setup.py demo.C build (gcc vs. g++) (distutils) In-Reply-To: <20070516000919.279273A4036@sparrow.telecommunity.com> References: <15086.1179025179@d.0x1.org> <475F4CA05E4E1B46A7213674E70EFA6202296938@SAUSEXMB2.amd.com> <4647DD2F.3000706@matt-good.net> <475F4CA05E4E1B46A7213674E70EFA620237D420@SAUSEXMB2.amd.com> <20070516000919.279273A4036@sparrow.telecommunity.com> Message-ID: <475F4CA05E4E1B46A7213674E70EFA620237D4A7@SAUSEXMB2.amd.com> set CC="g++" set CC=g++ Neither of these work :-( I also tried: python2.5 setup.py demo.C build --compiler=g++ python2.5 setup.py demo.C build --help-compiler gives bcpp, cygwin, emx, mingw32, msvc, mwerks, unix (g++ is not an option) Any other ideas? --Thanks -----Original Message----- From: Phillip J. Eby [mailto:pje at telecommunity.com] Sent: Tuesday, May 15, 2007 7:11 PM To: Mowry, Peter; Matt Good Cc: distutils-sig at python.org; David Arnold Subject: Re: [Distutils] setup.py demo.C build (gcc vs. g++) (distutils) At 06:11 PM 5/15/2007 -0500, Mowry, Peter wrote: >Content-class: urn:content-classes:message >Content-Type: multipart/related; > boundary="----_=_NextPart_001_01C79746.62B92D4C"; > type="multipart/alternative" > >Ever seen this error? > >Build Error Description: >python2.5 setup.py demo.C build > >That creates a demo.so, and when I try to import it, it gives me the error: >ImportError: ./demo.so: undefined symbol: __gxx_personality_v0 > > >Hack to fix it: >I found one single helpful post about this on google: >http://www.intevation.de/pipermail/thuban-list/2003-September/000230 .html >suggests: > >Do the link step of gdalwarp manually by copying the commandline >printed by setup.py and replacing the gcc with g++ >So I tried this, and I saw that setup.py prints the following 2 commands: >gcc -pthread -fno-strict-aliasing -DNDEBUG -g -O3 >-Wstrict-prototypes -fpermissive -fPIC >-I/tool/pandora64/.package/python-2.5/include/python2.5 -c demo.C -o >build/temp.linux-x86_64-2.5/demo.o >gcc -pthread -shared build/temp.linux-x86_64-2.5/demo.o -o >build/lib.linux-x86_64-2.5/demo.so > >Then I simply manually rerun just the second command with g++: >g++ -pthread -shared build/temp.linux-x86_64-2.5/demo.o -o >build/lib.linux-x86_64-2.5/demo.so > >And now the import works and so does the python spam_system() function. > > >Non-Hack to fix it?: >Is there a non-hack way to fix this? > >I've seen this question on the internet but no answers so far. I >did check --help-compiler, and it only lists --compiler=unix, not >--compiler=g++ for ex. I believe you can use the $CC environment variable, e.g. CC=g++ python setup.py whatever should cause it to use g++ as the executable instead of gcc. From hjebbers at gmail.com Wed May 16 11:53:16 2007 From: hjebbers at gmail.com (henk-jan ebbers) Date: Wed, 16 May 2007 11:53:16 +0200 Subject: [Distutils] import problem using setuptools (resend without html) In-Reply-To: <20070515211848.E3D9A3A4036@sparrow.telecommunity.com> References: <46499A27.3030703@gmail.com> <20070515151327.3E2073A4036@sparrow.telecommunity.com> <4649D3A2.8070905@gmail.com> <20070515162808.5E5413A4036@sparrow.telecommunity.com> <464A17B1.30804@gmail.com> <20070515211848.E3D9A3A4036@sparrow.telecommunity.com> Message-ID: <464AD48C.1010504@gmail.com> An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20070516/ba5473f6/attachment.html From pje at telecommunity.com Wed May 16 18:23:31 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 16 May 2007 12:23:31 -0400 Subject: [Distutils] setup.py demo.C build (gcc vs. g++) (distutils) In-Reply-To: <475F4CA05E4E1B46A7213674E70EFA620237D4A7@SAUSEXMB2.amd.com > References: <15086.1179025179@d.0x1.org> <475F4CA05E4E1B46A7213674E70EFA6202296938@SAUSEXMB2.amd.com> <4647DD2F.3000706@matt-good.net> <475F4CA05E4E1B46A7213674E70EFA620237D420@SAUSEXMB2.amd.com> <20070516000919.279273A4036@sparrow.telecommunity.com> <475F4CA05E4E1B46A7213674E70EFA620237D4A7@SAUSEXMB2.amd.com> Message-ID: <20070516162146.C46943A4036@sparrow.telecommunity.com> At 09:56 PM 5/15/2007 -0500, Mowry, Peter wrote: >set CC="g++" >set CC=g++ >Neither of these work :-( I may have misspoken; it may be the CXX variable you need to set, rather than CC. >I also tried: >python2.5 setup.py demo.C build --compiler=g++ > >python2.5 setup.py demo.C build --help-compiler >gives bcpp, cygwin, emx, mingw32, msvc, mwerks, unix (g++ is not an >option) That's because those are compiler *types*, not executables. To override the executables, you need to use environment variables like CC, CPP, CXX, CCSHARED, LDSHARED, etc. See the "customize_compiler()" function in the "distutils.sysconfig" module, as that's what sets the executable names that will be invoked. From peter.mowry at amd.com Wed May 16 19:19:41 2007 From: peter.mowry at amd.com (Mowry, Peter) Date: Wed, 16 May 2007 12:19:41 -0500 Subject: [Distutils] setup.py demo.C build (gcc vs. g++) (distutils) In-Reply-To: <20070516162146.C46943A4036@sparrow.telecommunity.com> References: <15086.1179025179@d.0x1.org> <475F4CA05E4E1B46A7213674E70EFA6202296938@SAUSEXMB2.amd.com> <4647DD2F.3000706@matt-good.net> <475F4CA05E4E1B46A7213674E70EFA620237D420@SAUSEXMB2.amd.com> <20070516000919.279273A4036@sparrow.telecommunity.com> <475F4CA05E4E1B46A7213674E70EFA620237D4A7@SAUSEXMB2.amd.com> <20070516162146.C46943A4036@sparrow.telecommunity.com> Message-ID: <475F4CA05E4E1B46A7213674E70EFA620237D774@SAUSEXMB2.amd.com> > echo $CC $CPP $CXX $LDSHARED g++ g++ g++ g++ > python2.5 setup.py build --compiler=unix running build running build_ext building 'demo' extension creating build creating build/temp.linux-x86_64-2.5 gcc -pthread -fno-strict-aliasing -DNDEBUG -g -O3 -Wstrict-prototypes -fpermissive -fPIC -I/tool/pandora64/.package/python-2.5/include/python2.5 -c demo.C -o build/temp.linux-x86_64-2.5/demo.o cc1plus: warning: command line option "-Wstrict-prototypes" is valid for C/ObjC but not for C++ creating build/lib.linux-x86_64-2.5 gcc -pthread -shared build/temp.linux-x86_64-2.5/demo.o -o build/lib.linux-x86_64-2.5/demo.so I saw CC CPP CXX LDSHARED - and set them all to g++, but it still does two calls to gcc, not g++ :-( --Thanks > setup.py: #!/tool/pandora64/bin/python2.5 from distutils.core import setup, Extension ext_demo = Extension('demo', sources = ['demo.C']) setup (name = 'PackageName', version = '1.0', description = 'This is a demo package', ext_modules = [ext_demo]) > demo.C: #include static PyObject* spam_system(PyObject *self, PyObject *args) { const char* command; int sts; if (!PyArg_ParseTuple(args, "s", &command)) return NULL; sts = system(command); return Py_BuildValue("i", sts); } static PyMethodDef demo_methods[] = { {"spam_system", spam_system, METH_VARARGS, "spam_system() doc string"}, {NULL, NULL} }; PyMODINIT_FUNC initdemo(void) { Py_InitModule("demo", demo_methods); } -----Original Message----- From: Phillip J. Eby [mailto:pje at telecommunity.com] Sent: Wednesday, May 16, 2007 11:24 AM To: Mowry, Peter; Matt Good Cc: distutils-sig at python.org; David Arnold Subject: RE: [Distutils] setup.py demo.C build (gcc vs. g++) (distutils) At 09:56 PM 5/15/2007 -0500, Mowry, Peter wrote: >set CC="g++" >set CC=g++ >Neither of these work :-( I may have misspoken; it may be the CXX variable you need to set, rather than CC. >I also tried: >python2.5 setup.py demo.C build --compiler=g++ > >python2.5 setup.py demo.C build --help-compiler >gives bcpp, cygwin, emx, mingw32, msvc, mwerks, unix (g++ is not an >option) That's because those are compiler *types*, not executables. To override the executables, you need to use environment variables like CC, CPP, CXX, CCSHARED, LDSHARED, etc. See the "customize_compiler()" function in the "distutils.sysconfig" module, as that's what sets the executable names that will be invoked. From pje at telecommunity.com Wed May 16 20:05:51 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 16 May 2007 14:05:51 -0400 Subject: [Distutils] setup.py demo.C build (gcc vs. g++) (distutils) In-Reply-To: <475F4CA05E4E1B46A7213674E70EFA620237D774@SAUSEXMB2.amd.com > References: <15086.1179025179@d.0x1.org> <475F4CA05E4E1B46A7213674E70EFA6202296938@SAUSEXMB2.amd.com> <4647DD2F.3000706@matt-good.net> <475F4CA05E4E1B46A7213674E70EFA620237D420@SAUSEXMB2.amd.com> <20070516000919.279273A4036@sparrow.telecommunity.com> <475F4CA05E4E1B46A7213674E70EFA620237D4A7@SAUSEXMB2.amd.com> <20070516162146.C46943A4036@sparrow.telecommunity.com> <475F4CA05E4E1B46A7213674E70EFA620237D774@SAUSEXMB2.amd.com> Message-ID: <20070516180404.B537D3A4036@sparrow.telecommunity.com> At 12:19 PM 5/16/2007 -0500, Mowry, Peter wrote: > > echo $CC $CPP $CXX $LDSHARED >g++ g++ g++ g++ Try this instead: python2.5 -c "import os; print os.environ['CXX']" Does it display the same thing? From peter.mowry at amd.com Wed May 16 20:33:56 2007 From: peter.mowry at amd.com (Mowry, Peter) Date: Wed, 16 May 2007 13:33:56 -0500 Subject: [Distutils] setup.py demo.C build (gcc vs. g++) (distutils) In-Reply-To: <20070516180404.B537D3A4036@sparrow.telecommunity.com> References: <15086.1179025179@d.0x1.org> <475F4CA05E4E1B46A7213674E70EFA6202296938@SAUSEXMB2.amd.com> <4647DD2F.3000706@matt-good.net> <475F4CA05E4E1B46A7213674E70EFA620237D420@SAUSEXMB2.amd.com> <20070516000919.279273A4036@sparrow.telecommunity.com> <475F4CA05E4E1B46A7213674E70EFA620237D4A7@SAUSEXMB2.amd.com> <20070516162146.C46943A4036@sparrow.telecommunity.com> <475F4CA05E4E1B46A7213674E70EFA620237D774@SAUSEXMB2.amd.com> <20070516180404.B537D3A4036@sparrow.telecommunity.com> Message-ID: <475F4CA05E4E1B46A7213674E70EFA620237D7FC@SAUSEXMB2.amd.com> You're right, the environment variables weren't getting imported. So I just added it to my python code instead: import os; os.environ['CC'] = 'g++'; os.environ['CXX'] = 'g++'; os.environ['CPP'] = 'g++'; os.environ['LDSHARED'] = 'g++' I found that CC determined the 1st command and LDSHARED determined the second one (gcc vs. g++). But strangely enough, neither (CC=g++ and LDSHARED=g++) or (CC=g++ and LDSHARED=gcc) worked. But if I do (CC=gcc and LDSHARED=gcc) then manually redo the 2nd g++, then it works. See below... Seems to be some kind of linking issue... (Thanks) > python2.5 setup.py build --compiler=unix running build running build_ext building 'demo' extension creating build creating build/temp.linux-x86_64-2.5 g++ -fno-strict-aliasing -DNDEBUG -g -O3 -Wstrict-prototypes -fpermissive -fPIC -I/tool/pandora64/.package/python-2.5/include/python2.5 -c demo.C -o build/temp.linux-x86_64-2.5/demo.o cc1plus: warning: command line option "-Wstrict-prototypes" is valid for C/ObjC but not for C++ creating build/lib.linux-x86_64-2.5 g++ build/temp.linux-x86_64-2.5/demo.o -o build/lib.linux-x86_64-2.5/demo.so /usr/lib/../lib64/crt1.o: In function `_start': /usr/lib/../lib64/crt1.o(.text+0x21): undefined reference to `main' build/temp.linux-x86_64-2.5/demo.o: In function `spam_system(_object*, _object*)': initdemo/demo.C:8: undefined reference to `PyArg_ParseTuple' initdemo/demo.C:11: undefined reference to `Py_BuildValue' build/temp.linux-x86_64-2.5/demo.o: In function `initdemo': initdemo/demo.C:22: undefined reference to `Py_InitModule4_64' collect2: ld returned 1 exit status error: command 'g++' failed with exit status 1 > python2.5 setup.py build --compiler=unix running build running build_ext building 'demo' extension creating build creating build/temp.linux-x86_64-2.5 gcc -pthread -fno-strict-aliasing -DNDEBUG -g -O3 -Wstrict-prototypes -fpermissive -fPIC -I/tool/pandora64/.package/python-2.5/include/python2.5 -c demo.C -o build/temp.linux-x86_64-2.5/demo.o cc1plus: warning: command line option "-Wstrict-prototypes" is valid for C/ObjC but not for C++ creating build/lib.linux-x86_64-2.5 g++ build/temp.linux-x86_64-2.5/demo.o -o build/lib.linux-x86_64-2.5/demo.so /usr/lib/../lib64/crt1.o: In function `_start': /usr/lib/../lib64/crt1.o(.text+0x21): undefined reference to `main' build/temp.linux-x86_64-2.5/demo.o: In function `spam_system(_object*, _object*)': initdemo/demo.C:8: undefined reference to `PyArg_ParseTuple' initdemo/demo.C:11: undefined reference to `Py_BuildValue' build/temp.linux-x86_64-2.5/demo.o: In function `initdemo': initdemo/demo.C:22: undefined reference to `Py_InitModule4_64' collect2: ld returned 1 exit status error: command 'g++' failed with exit status 1 > python2.5 setup.py build --compiler=unix running build running build_ext building 'demo' extension creating build creating build/temp.linux-x86_64-2.5 gcc -pthread -fno-strict-aliasing -DNDEBUG -g -O3 -Wstrict-prototypes -fpermissive -fPIC -I/tool/pandora64/.package/python-2.5/include/python2.5 -c demo.C -o build/temp.linux-x86_64-2.5/demo.o cc1plus: warning: command line option "-Wstrict-prototypes" is valid for C/ObjC but not for C++ creating build/lib.linux-x86_64-2.5 gcc -pthread -shared build/temp.linux-x86_64-2.5/demo.o -o build/lib.linux-x86_64-2.5/demo.so > g++ -pthread -shared build/temp.linux-x86_64-2.5/demo.o -o build/lib.linux-x86_64-2.5/demo.so > echo "/cheer - that works - I can import the module in python" -----Original Message----- From: Phillip J. Eby [mailto:pje at telecommunity.com] Sent: Wednesday, May 16, 2007 1:06 PM To: Mowry, Peter; Matt Good Cc: distutils-sig at python.org; David Arnold Subject: RE: [Distutils] setup.py demo.C build (gcc vs. g++) (distutils) At 12:19 PM 5/16/2007 -0500, Mowry, Peter wrote: > > echo $CC $CPP $CXX $LDSHARED >g++ g++ g++ g++ Try this instead: python2.5 -c "import os; print os.environ['CXX']" Does it display the same thing? From jmg3000 at gmail.com Wed May 16 20:59:06 2007 From: jmg3000 at gmail.com (John Gabriele) Date: Wed, 16 May 2007 14:59:06 -0400 Subject: [Distutils] keeping a local ~/pylib directory of installed modules (2) Message-ID: <65e0bb520705161159v69079fb4u5246d4942cd3df6e@mail.gmail.com> I wrote: > I'm running on a server where I don't have root access. I've installed > my own Python into ~/py-2.5.1 (with a ~/py symlink), and created my > own ~/pylib directory for installing any extra packages I need. > > I'd like to start using Easy Install / setuptools to manage package > installation, and have read the "Custom Installation Locations" --> " Ok, I see. If you've got your own local Python installed that you can write to, and that python is the first on your $PATH, everything just works with no extra options and things just get installed into your own local ``lib/python2.n/site-packages`` dir. Don't need a ~/pylib for an all-local setup like this. ---John From b.n.lawrence at rl.ac.uk Fri May 18 18:15:40 2007 From: b.n.lawrence at rl.ac.uk (Bryan Lawrence) Date: Fri, 18 May 2007 17:15:40 +0100 Subject: [Distutils] easy_install and sys.path In-Reply-To: <20070509174007.A81D13A409D@sparrow.telecommunity.com> References: <200705081627.17538.b.n.lawrence@rl.ac.uk> <200705090825.39865.b.n.lawrence@rl.ac.uk> <20070509174007.A81D13A409D@sparrow.telecommunity.com> Message-ID: <200705181715.40648.b.n.lawrence@rl.ac.uk> Hi Phillip Thanks for this. For the record it seems that debian has patched the site.py to include the /usr/local tree (and hence all the ubuntu family have this as well). So, it would appear for Debian it isn't possible to have your own python in /usr/local at the same version as the system python on a debian system ... and expect them to be independent. Damn. I appear to have got away with this in the past because either a) I've had a different version, or b) I haven't had eggs (or both). Obviously there are ways around this, one of which you've outlined below (again, thanks), but it's not very tidy (which you can translate as me not understanding why they've done this, not complaining about easy_install :-) Cheers Bryan On Wednesday 09 May 2007 18:41:52 Phillip J. Eby wrote: > At 08:25 AM 5/9/2007 +0100, Bryan Lawrence wrote: > >Oh I know I shouldn't mix two pythons, but I've never had them "mixed" > > before. (We have run our own python in /usr/local for yonks without any > > bad karma until now). It's only the advent of eggs, which push things to > > the top of sys.path that causes a problem. It would appear that I'm not > > the only person who has the problem (e.g. see > >http://www.rexx.com/~dkuhlman/python_comments.html). > > > >So, my question to this list is "why"? > > > >Why does easy_install put things in the front of sys.path? > >Presumably there is > >some good reason (perhaps it is the only way to ensure that your > > dependency libraries are the ones that are picked up, I guess there is a > > risk that older libraries might exist > > That's precisely the reason. Until it started doing this, it had to > try to either delete those conflicting libraries, or refuse to > install anything until the user fixed it. > > >I've been a big fan of eggs, but I've just slammed into a wall. I can't > >believe that I'm the only one for whom this may be a problem. I now need > > to work out what, if anything, I need to do about it (change my > > behaviour, find out something I didn't know about egg installation, > > provide a suggested optional change to egg behaviour, go home have a > > drink etc ...) > > You can use the "-m" option to easy_install, in which case eggs will > only be added to sys.path when you ask for them. Just delete your > existing easy-install.pth file (leaving the setuptools.pth file in > place). Then, your existing eggs will still be there, and any > easy_install-provided scripts will still work. You just will have to > manually require() any eggs you want when you run any program that's > not a setuptools-generated script, or when you fire up the > interpreter (unless you modify your .pythonrc to require() the eggs you > want). > > > > '/usr/local/lib/python2.5/site-packages', > > The fact that this is in your sys.path is what's causing those eggs > to be added, btw. It appears your distro may have either patched > /usr/lib/python2.5/site.py to include the /usr/local tree, or you > have a .pth file in /usr/lib/python2.5/site-packages that adds the > /usr/local/.../site-packages directory. From jim at zope.com Fri May 18 18:43:15 2007 From: jim at zope.com (Jim Fulton) Date: Fri, 18 May 2007 12:43:15 -0400 Subject: [Distutils] Overly picky about file URLs In-Reply-To: <20070515231012.077633A4036@sparrow.telecommunity.com> References: <80498EC4-E2E5-42E2-BA3F-A25CBB7B4200@zope.com> <20070515174651.7E2113A4036@sparrow.telecommunity.com> <17846F4A-F132-407E-BEC2-6A593F1B25F5@zope.com> <20070515231012.077633A4036@sparrow.telecommunity.com> Message-ID: On May 15, 2007, at 7:12 PM, Phillip J. Eby wrote: > The relative URLs that are in the HTML that local_open() returns > will be broken if you do that. Ah, I see you're point. The leading '/'s would need to be added upstream of the local_open call. When I get a chance I'll see if I can make a better suggestion. I'm afraid this isn't very urgent for me atm, as I have a work around in buildout. Jim -- Jim Fulton mailto:jim at zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From jim at zope.com Fri May 18 18:51:05 2007 From: jim at zope.com (Jim Fulton) Date: Fri, 18 May 2007 12:51:05 -0400 Subject: [Distutils] setuptools process_url doesn't handle URLErrors gracefullly In-Reply-To: <20070515174827.ECD2E3A4036@sparrow.telecommunity.com> References: <1178871702.22778.8.camel@mindy.whq.gocept.com> <20070515174827.ECD2E3A4036@sparrow.telecommunity.com> Message-ID: <1EF5ED10-6ADD-4355-9B06-04C9E3B69150@zope.com> On May 15, 2007, at 1:50 PM, Phillip J. Eby wrote: > At 09:52 AM 5/14/2007 -0400, Jim Fulton wrote: >> This problem is rather serious. As we rely more and more on >> setuptools, it needs to be robust in the presence of this sort of >> failure. >> >> Phillip, If I fix this, can we get a timely release with the fix? > > Is sometime next week sufficient to be described as timely? If so, > then yes. Yes, great, sort of. Now I have to find time to actually make the fix. I'll try. :) Jim -- Jim Fulton mailto:jim at zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From pje at telecommunity.com Fri May 18 19:20:07 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 18 May 2007 13:20:07 -0400 Subject: [Distutils] Overly picky about file URLs In-Reply-To: References: <80498EC4-E2E5-42E2-BA3F-A25CBB7B4200@zope.com> <20070515174651.7E2113A4036@sparrow.telecommunity.com> <17846F4A-F132-407E-BEC2-6A593F1B25F5@zope.com> <20070515231012.077633A4036@sparrow.telecommunity.com> Message-ID: <20070518171818.3A6083A4061@sparrow.telecommunity.com> At 12:43 PM 5/18/2007 -0400, Jim Fulton wrote: >On May 15, 2007, at 7:12 PM, Phillip J. Eby wrote: >>The relative URLs that are in the HTML that local_open() returns >>will be broken if you do that. > >Ah, I see you're point. The leading '/'s would need to be added >upstream of the local_open call. Or else the directory name and a '/' would need to be added in local_open(), as I mentioned before. From tseaver at palladion.com Fri May 18 20:32:44 2007 From: tseaver at palladion.com (Tres Seaver) Date: Fri, 18 May 2007 14:32:44 -0400 Subject: [Distutils] easy_install and sys.path In-Reply-To: <200705181715.40648.b.n.lawrence@rl.ac.uk> References: <200705081627.17538.b.n.lawrence@rl.ac.uk> <200705090825.39865.b.n.lawrence@rl.ac.uk> <20070509174007.A81D13A409D@sparrow.telecommunity.com> <200705181715.40648.b.n.lawrence@rl.ac.uk> Message-ID: <464DF14C.7030901@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Bryan Lawrence wrote: > Hi Phillip > > Thanks for this. > > For the record it seems that debian has patched the site.py to include > the /usr/local tree (and hence all the ubuntu family have this as well). > > So, it would appear for Debian it isn't possible to have your own python > in /usr/local at the same version as the system python on a debian system ... > and expect them to be independent. Damn. I appear to have got away with this > in the past because either a) I've had a different version, or b) I haven't > had eggs (or both). > > Obviously there are ways around this, one of which you've outlined below > (again, thanks), but it's not very tidy (which you can translate as me not > understanding why they've done this, not complaining about easy_install :-) It would appear that their justification is that '/usr/local/lib/python2.x' is where they expect non-packaged extensions to be placed; never mind that there is no facility within Python itself to install such extensions into that place. Essentially, if you want to install a different Python than /usr/bin/python2.x on a Debian box, you need to put it somewhere besides '/usr/local' (e.g., perhaps '/opt/somename'). I would class this as a bug in the Debian packaging of Python, but have no intention of arguing with them about the correctness of their choice. One thing you might try is to have a 'sitecustomize.py' which removed the '/usr/local' entries from sys.path; that appears to be the "last" hook point available. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGTfFM+gerLs4ltQ4RArKgAJ9vO814yLEtRFZVwNDGfA39PEsjcgCeNzBo NmFzig6N137SjiDVAWCl1oM= =SRse -----END PGP SIGNATURE----- From tseaver at palladion.com Fri May 18 20:32:44 2007 From: tseaver at palladion.com (Tres Seaver) Date: Fri, 18 May 2007 14:32:44 -0400 Subject: [Distutils] easy_install and sys.path In-Reply-To: <200705181715.40648.b.n.lawrence@rl.ac.uk> References: <200705081627.17538.b.n.lawrence@rl.ac.uk> <200705090825.39865.b.n.lawrence@rl.ac.uk> <20070509174007.A81D13A409D@sparrow.telecommunity.com> <200705181715.40648.b.n.lawrence@rl.ac.uk> Message-ID: <464DF14C.7030901@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Bryan Lawrence wrote: > Hi Phillip > > Thanks for this. > > For the record it seems that debian has patched the site.py to include > the /usr/local tree (and hence all the ubuntu family have this as well). > > So, it would appear for Debian it isn't possible to have your own python > in /usr/local at the same version as the system python on a debian system ... > and expect them to be independent. Damn. I appear to have got away with this > in the past because either a) I've had a different version, or b) I haven't > had eggs (or both). > > Obviously there are ways around this, one of which you've outlined below > (again, thanks), but it's not very tidy (which you can translate as me not > understanding why they've done this, not complaining about easy_install :-) It would appear that their justification is that '/usr/local/lib/python2.x' is where they expect non-packaged extensions to be placed; never mind that there is no facility within Python itself to install such extensions into that place. Essentially, if you want to install a different Python than /usr/bin/python2.x on a Debian box, you need to put it somewhere besides '/usr/local' (e.g., perhaps '/opt/somename'). I would class this as a bug in the Debian packaging of Python, but have no intention of arguing with them about the correctness of their choice. One thing you might try is to have a 'sitecustomize.py' which removed the '/usr/local' entries from sys.path; that appears to be the "last" hook point available. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGTfFM+gerLs4ltQ4RArKgAJ9vO814yLEtRFZVwNDGfA39PEsjcgCeNzBo NmFzig6N137SjiDVAWCl1oM= =SRse -----END PGP SIGNATURE----- From peter.mowry at amd.com Fri May 18 20:33:50 2007 From: peter.mowry at amd.com (Mowry, Peter) Date: Fri, 18 May 2007 13:33:50 -0500 Subject: [Distutils] build python2.5 extension on windows64 with MSVC 8 - _MSVCCompiler__root Message-ID: <475F4CA05E4E1B46A7213674E70EFA620241DE56@SAUSEXMB2.amd.com> C:\Pem\simnow_checkouts\simnow_B\PEM>C:\Python25\python.exe setup.py build --compiler=msvc running build running build_ext building 'demo' extension Traceback (most recent call last): File "setup.py", line 20, in ext_modules = [ext_demo]) File "C:\Python25\lib\distutils\core.py", line 151, in setup dist.run_commands() File "C:\Python25\lib\distutils\dist.py", line 974, in run_commands self.run_command(cmd) File "C:\Python25\lib\distutils\dist.py", line 994, in run_command cmd_obj.run() File "C:\Python25\lib\distutils\command\build.py", line 112, in run self.run_command(cmd_name) File "C:\Python25\lib\distutils\cmd.py", line 333, in run_command self.distribution.run_command(command) File "C:\Python25\lib\distutils\dist.py", line 994, in run_command cmd_obj.run() File "C:\Python25\lib\distutils\command\build_ext.py", line 290, in run self.build_extensions() File "C:\Python25\lib\distutils\command\build_ext.py", line 416, in build_extensions self.build_extension(ext) File "C:\Python25\lib\distutils\command\build_ext.py", line 481, in build_extension depends=ext.depends) File "C:\Python25\lib\distutils\msvccompiler.py", line 354, in compile if not self.initialized: self.initialize() File "C:\Python25\lib\distutils\msvccompiler.py", line 264, in initialize self.__paths = self.get_msvc_paths("path") File "C:\Python25\lib\distutils\msvccompiler.py", line 616, in get_msvc_paths % (self.__root, self.__version)) AttributeError: MSVCCompiler instance has no attribute '_MSVCCompiler__root' I did some web searching and read that I can: os.environ['DISTUTILS_USE_SDK'] = "1" os.environ['MSSdk'] = "1" http://docs.python.org/dist/module-distutils.msvccompiler.html C:\Pem\simnow_checkouts\simnow_B\PEM>C:\Python25\python.exe setup.py build --compiler=msvc running build running build_ext building 'demo' extension error: Don't know how to compile demo.C Unfortunately it sounds like I might need to use the "Platform SDK" to compile even my very basic python extension module (just loads a dll and calls one method) (Windows64, python2.5) :-( -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20070518/55e54f3d/attachment-0001.htm From theller at ctypes.org Fri May 18 20:51:09 2007 From: theller at ctypes.org (Thomas Heller) Date: Fri, 18 May 2007 20:51:09 +0200 Subject: [Distutils] build python2.5 extension on windows64 with MSVC 8 - _MSVCCompiler__root In-Reply-To: <475F4CA05E4E1B46A7213674E70EFA620241DE56@SAUSEXMB2.amd.com> References: <475F4CA05E4E1B46A7213674E70EFA620241DE56@SAUSEXMB2.amd.com> Message-ID: Mowry, Peter schrieb: You cannot use MSVC8 anyway because it links with the wrong C runtime library; except if you compile thw whole Python with it. > > I did some web searching and read that I can: > os.environ['DISTUTILS_USE_SDK'] = "1" > os.environ['MSSdk'] = "1" > > http://docs.python.org/dist/module-distutils.msvccompiler.html > > Unfortunately it sounds like I might need to use the "Platform SDK" to > compile even my very basic python extension module (just loads a dll and > calls one method) (Windows64, python2.5) :-( So what? As an alternative you might try out ctypes, which I just released for win64. Thomas From peter.mowry at amd.com Fri May 18 23:10:41 2007 From: peter.mowry at amd.com (Mowry, Peter) Date: Fri, 18 May 2007 16:10:41 -0500 Subject: [Distutils] Compile Windows64 AMD64 Message-ID: <475F4CA05E4E1B46A7213674E70EFA620241E001@SAUSEXMB2.amd.com> I am trying to compile a python extension for Windows64 using distutils from "Python 2.5.1 (r251:54863, Apr 18 2007, 09:02:36) [MSC v.1400 64 bit (AMD64)] on win32". My extension basically is just a simple C code file which does a LoadLibrary (dlopen for unix) of my 64-bit shared library (dll/so) (and then call one function to test it). My python extension compiled and ran on unix, but not windows. Originally it says: File "C:\Python25\lib\distutils\msvccompiler.py", line 616, in get_msvc_paths AttributeError: MSVCCompiler instance has no attribute '_MSVCCompiler__root' Then when I try it after setting env variables DISTUTILS_USE_SDK and MSSdk (both to "1"), it says: error: Don't know how to compile demo.C ditutils doc (dist.pdf) section 11.4 says: Typically, extension modules need to be compiled with the same compiler that was used to compile Python. ... The AMD64 and Itanium binaries are created using the Platform SDK. I am using Visual Studios 8 normally, but I downloaded the Platform SDK 2003 SP1. I opened the start menu "Set Windows XP x64 Build Environment (Retail)", and tried building it from there. This made no difference; I got the same errors. Not fun at all :-( -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20070518/10746c67/attachment.htm From jim at zope.com Sat May 19 00:01:16 2007 From: jim at zope.com (Jim Fulton) Date: Fri, 18 May 2007 18:01:16 -0400 Subject: [Distutils] setuptools process_url doesn't handle URLErrors gracefullly In-Reply-To: <1178871702.22778.8.camel@mindy.whq.gocept.com> References: <1178871702.22778.8.camel@mindy.whq.gocept.com> Message-ID: <060B7813-C06D-4A05-93E0-9774C36384D2@zope.com> This is mostly for Phillip, or anyone else who might have a way to guess how the impossible might happen, possibly including experts in the way that urllib2 works. :) On May 11, 2007, at 4:21 AM, Christian Theune wrote: > I'm fighting against a problem where the DNS name of the homepage > of a > package that is listed on the cheeseshop isn't accessible any more. In > this special case the DNS name for this thing is gone (the package is > `rwproperty` the missing DNS entry is z3lab.org). > > In this case easy_install dies with the following error, although the > eggs are listed on the cheeseshop page and the listed homepage is > pretty > irrelevant: > > Searching for rwproperty > Reading http://cheeseshop.python.org/pypi/rwproperty/ > Reading > http://www.z3lab.org/sections/blogs/philipp-weitershausen/ > 2006_05_29_pycon-06-lightning-talk > Traceback (most recent call last): > File "/usr/bin/easy_install", line 7, in ? > sys.exit( > File > "/usr/lib/python2.4/site-packages/setuptools-0.6c5-py2.4.egg/ > setuptools/command/easy_install.py", line 1670, in main > with_ei_usage(lambda: > File > "/usr/lib/python2.4/site-packages/setuptools-0.6c5-py2.4.egg/ > setuptools/command/easy_install.py", line 1659, in with_ei_usage > return f() > File > "/usr/lib/python2.4/site-packages/setuptools-0.6c5-py2.4.egg/ > setuptools/command/easy_install.py", line 1674, in > distclass=DistributionWithoutHelpCommands, **kw > File "/usr/lib/python2.4/distutils/core.py", line 149, in setup > dist.run_commands() > File "/usr/lib/python2.4/distutils/dist.py", line 946, in > run_commands > self.run_command(cmd) > File "/usr/lib/python2.4/distutils/dist.py", line 966, in > run_command > cmd_obj.run() > File > "/usr/lib/python2.4/site-packages/setuptools-0.6c5-py2.4.egg/ > setuptools/command/easy_install.py", line 211, in run > self.easy_install(spec, not self.no_deps) > File > "/usr/lib/python2.4/site-packages/setuptools-0.6c5-py2.4.egg/ > setuptools/command/easy_install.py", line 432, in easy_install > dist = self.package_index.fetch_distribution( > File > "/usr/lib/python2.4/site-packages/setuptools-0.6c5-py2.4.egg/ > setuptools/package_index.py", line 462, in fetch_distribution > self.find_packages(requirement) > File > "/usr/lib/python2.4/site-packages/setuptools-0.6c5-py2.4.egg/ > setuptools/package_index.py", line 303, in find_packages > self.scan_url(self.index_url + requirement.unsafe_name+'/') > File > "/usr/lib/python2.4/site-packages/setuptools-0.6c5-py2.4.egg/ > setuptools/package_index.py", line 610, in scan_url > self.process_url(url, True) > File > "/usr/lib/python2.4/site-packages/setuptools-0.6c5-py2.4.egg/ > setuptools/package_index.py", line 201, in process_url > page = self.process_index(url, page) > File > "/usr/lib/python2.4/site-packages/setuptools-0.6c5-py2.4.egg/ > setuptools/package_index.py", line 278, in process_index > self.scan_url(new_url) > File > "/usr/lib/python2.4/site-packages/setuptools-0.6c5-py2.4.egg/ > setuptools/package_index.py", line 610, in scan_url > self.process_url(url, True) > File > "/usr/lib/python2.4/site-packages/setuptools-0.6c5-py2.4.egg/ > setuptools/package_index.py", line 191, in process_url > self.fetched_urls[url] = self.fetched_urls[f.url] = True > AttributeError: URLError instance has no attribute 'url' This particular traceback is very very weird. The line before the error line computes f by calling open_url: f = self.open_url(url) open_url is: def open_url(self, url): if url.startswith('file:'): return local_open(url) try: return open_with_auth(url) except urllib2.HTTPError, v: return v except urllib2.URLError, v: raise DistutilsError("Download error: %s" % v.reason) open_with_auth returns the result of calling urllib2.urlopen. If urlopen raised URLError, it would have been caught by open_url and re-raised as a DistutilsError. The only way I can see open_url *returning* a URLError is if something in urllib2 returned it and I don't see that anywhere. I suppose that something could be catching URLError and returning it, but I don't see any evidence of that either. I also suppose that there could be some weird urllib2 plugin that is causing this behavior. The original server problem that spurred this report has been fixed. When I try to reproduce this myself, using a faux index server and project, I get the DistutilsError I would expect to be raised by open_url. Also, buildbot logs we have from the original server problem on our servers also show the expected DistutilsError. I'm going to work on a fix assuming that open_url is raising a DistutilsError in this situation. I'm most puzzled and troubled by Christian's traceback. My fix won't fix his symptom, and I don't want to try to hack in a fix for his symptom if I can't reproduce or test it. If anyone else has any insights, they'd be welcome. :) My last hope is that the test I write for the fix will fail on Christian's machine. Jim -- Jim Fulton mailto:jim at zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From theller at ctypes.org Sat May 19 00:07:52 2007 From: theller at ctypes.org (Thomas Heller) Date: Sat, 19 May 2007 00:07:52 +0200 Subject: [Distutils] Compile Windows64 AMD64 In-Reply-To: <475F4CA05E4E1B46A7213674E70EFA620241E001@SAUSEXMB2.amd.com> References: <475F4CA05E4E1B46A7213674E70EFA620241E001@SAUSEXMB2.amd.com> Message-ID: Mowry, Peter schrieb: > I am trying to compile a python extension for Windows64 using distutils > from "Python 2.5.1 (r251:54863, Apr 18 2007, 09:02:36) [MSC v.1400 64 > bit (AMD64)] on win32". My extension basically is just a simple C code > file which does a LoadLibrary (dlopen for unix) of my 64-bit shared > library (dll/so) (and then call one function to test it). > > Then when I try it after setting env variables DISTUTILS_USE_SDK and > MSSdk (both to "1"), it says: > > error: Don't know how to compile demo.C > Can it be that the file must be named 'demo.c' instead of 'demo.C'? Thomas From peter.mowry at amd.com Sat May 19 00:19:54 2007 From: peter.mowry at amd.com (Mowry, Peter) Date: Fri, 18 May 2007 17:19:54 -0500 Subject: [Distutils] Compile Windows64 AMD64 Message-ID: <475F4CA05E4E1B46A7213674E70EFA620241E069@SAUSEXMB2.amd.com> So I tried ctypes, and it just worked (without any build issues - in fact I didn't have to build anything or write any code). Is there any reason I'd want to write an extend python module (to load my shared library and run exported functions from it) that compiles with distutils, instead of just using ctypes? Thanks ________________________________ From: Mowry, Peter Sent: Friday, May 18, 2007 4:11 PM To: 'distutils-sig at python.org' Subject: Compile Windows64 AMD64 I am trying to compile a python extension for Windows64 using distutils from "Python 2.5.1 (r251:54863, Apr 18 2007, 09:02:36) [MSC v.1400 64 bit (AMD64)] on win32". My extension basically is just a simple C code file which does a LoadLibrary (dlopen for unix) of my 64-bit shared library (dll/so) (and then call one function to test it). My python extension compiled and ran on unix, but not windows. Originally it says: File "C:\Python25\lib\distutils\msvccompiler.py", line 616, in get_msvc_paths AttributeError: MSVCCompiler instance has no attribute '_MSVCCompiler__root' Then when I try it after setting env variables DISTUTILS_USE_SDK and MSSdk (both to "1"), it says: error: Don't know how to compile demo.C ditutils doc (dist.pdf) section 11.4 says: Typically, extension modules need to be compiled with the same compiler that was used to compile Python. ... The AMD64 and Itanium binaries are created using the Platform SDK. I am using Visual Studios 8 normally, but I downloaded the Platform SDK 2003 SP1. I opened the start menu "Set Windows XP x64 Build Environment (Retail)", and tried building it from there. This made no difference; I got the same errors. Not fun at all :-( -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20070518/9c033feb/attachment.html From clovisgo at gmail.com Sat May 19 15:57:35 2007 From: clovisgo at gmail.com (Clovis Goldemberg) Date: Sat, 19 May 2007 10:57:35 -0300 Subject: [Distutils] Silent install of Python modules. Message-ID: <6f239f130705190657h7261ab7cu3531660396e02a1a@mail.gmail.com> Suppose I have a bunch of pre-built packages called: XXX1-1.0.0.win32.exe XXX2-1.0.0.win32.exe ... XXXN-1.0.0.win32.exe Is it possible to "automate" the installation, without the need of pressing "Next" so many times? The main idea would be an installation option like "-q" (quiet) ou "-s" (silent). So if I call: XXX1-1.0.0.win32.exe -q XXX2-1.0.0.win32.exe -q ... XXXN-1.0.0.win32.exe -q this would do the job "quietly". The user intervention would be only required if something goes wrong. Thanks for all. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20070519/f46ab9ba/attachment.htm From lxander.m at gmail.com Sun May 20 14:03:35 2007 From: lxander.m at gmail.com (Alexander Michael) Date: Sun, 20 May 2007 08:03:35 -0400 Subject: [Distutils] Compile Windows64 AMD64 In-Reply-To: <475F4CA05E4E1B46A7213674E70EFA620241E069@SAUSEXMB2.amd.com> References: <475F4CA05E4E1B46A7213674E70EFA620241E069@SAUSEXMB2.amd.com> Message-ID: <525f23e80705200503i4bbba299qd03bf6056b9ad89a@mail.gmail.com> On 5/18/07, Mowry, Peter wrote: > > So I tried ctypes, and it just worked (without any build issues - in fact > I didn't have to build anything or write any code). Is there any reason I'd > want to write an extend python module (to load my shared library and run > exported functions from it) that compiles with distutils, instead of just > using ctypes? Thanks > If I can accomplish the task with ctypes, I use it. While distutils generally does a good job with compiling extension modules, using ctypes allows you to avoid distributing platform specific packages which will make your life a bit easier. Additionally, I find it faster to work with ctypes as I can avoid the compile step required for developing extension module. While ctypes is part of Python 2.5, your users will need to install it for Python 2.3 and 2.4. If you need to target earlier versions of Python (*not as typical these days), then ctypes won't work for you. Unfortunately, ctypes is not available as an egg, so you won't be able to take advantage of setuptool's easy_install to ensure that your user has installed all the dependencies. Alex -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20070520/b87d7f7c/attachment.html From lxander.m at gmail.com Sun May 20 14:23:01 2007 From: lxander.m at gmail.com (Alexander Michael) Date: Sun, 20 May 2007 08:23:01 -0400 Subject: [Distutils] Compile Windows64 AMD64 In-Reply-To: <525f23e80705200503i4bbba299qd03bf6056b9ad89a@mail.gmail.com> References: <475F4CA05E4E1B46A7213674E70EFA620241E069@SAUSEXMB2.amd.com> <525f23e80705200503i4bbba299qd03bf6056b9ad89a@mail.gmail.com> Message-ID: <525f23e80705200523n1ea440ebhc252d43ae5e52882@mail.gmail.com> On 5/20/07, Alexander Michael wrote: > > On 5/18/07, Mowry, Peter wrote: > > > > So I tried ctypes, and it just worked (without any build issues - in > > fact I didn't have to build anything or write any code). Is there any > > reason I'd want to write an extend python module (to load my shared library > > and run exported functions from it) that compiles with distutils, instead of > > just using ctypes? Thanks > > > If I can accomplish the task with ctypes, I use it. While distutils > generally does a good job with compiling extension modules, using ctypes > allows you to avoid distributing platform specific packages which will make > your life a bit easier. Additionally, I find it faster to work with ctypes > as I can avoid the compile step required for developing extension module. > > While ctypes is part of Python 2.5, your users will need to install it for > Python 2.3 and 2.4. If you need to target earlier versions of Python (*not > as typical these days), then ctypes won't work for you. Unfortunately, > ctypes is not available as an egg, so you won't be able to take advantage of > setuptool's easy_install to ensure that your user has installed all the > dependencies. > Correction: Apparently easy_install handles source packages that aren't eggs, so scratch my scratch comment as it looks like you can specify ctypes as a dependency in order to ensure that it is installed. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20070520/f742febd/attachment.htm From mhammond at skippinet.com.au Sun May 20 15:19:21 2007 From: mhammond at skippinet.com.au (Mark Hammond) Date: Sun, 20 May 2007 23:19:21 +1000 Subject: [Distutils] Compile Windows64 AMD64 In-Reply-To: <475F4CA05E4E1B46A7213674E70EFA620241E069@SAUSEXMB2.amd.com> Message-ID: <002301c79ae1$7b623020$1f0a0a0a@enfoldsystems.local> > So I tried ctypes, and it just worked (without any build issues - in fact I didn't > have to build anything or write any code). Is there any reason I'd want to write > an extend python module (to load my shared library and run exported functions from >it) that compiles with distutils, instead of just using ctypes? If your extension module does exactly what you describe - ie, load functions from a DLL and calls them, then ctypes is a perfect replacement. Mark -------------- next part -------------- A non-text attachment was scrubbed... Name: winmail.dat Type: application/ms-tnef Size: 1804 bytes Desc: not available Url : http://mail.python.org/pipermail/distutils-sig/attachments/20070520/c2825f1e/attachment.bin From pje at telecommunity.com Sun May 20 16:53:27 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 20 May 2007 10:53:27 -0400 Subject: [Distutils] Compile Windows64 AMD64 In-Reply-To: <525f23e80705200503i4bbba299qd03bf6056b9ad89a@mail.gmail.co m> References: <475F4CA05E4E1B46A7213674E70EFA620241E069@SAUSEXMB2.amd.com> <525f23e80705200503i4bbba299qd03bf6056b9ad89a@mail.gmail.com> Message-ID: <20070520145143.2FB6F3A4061@sparrow.telecommunity.com> At 08:03 AM 5/20/2007 -0400, Alexander Michael wrote: >Unfortunately, ctypes is not available as an egg, so you won't be >able to take advantage of setuptool's easy_install to ensure that >your user has installed all the dependencies. easy_install converts distutils .exe files to eggs just fine, and it compiles the source on other platforms. Windows: $ python2.4 -m easy_install -U ctypes Searching for ctypes Reading http://cheeseshop.python.org/pypi/ctypes/ Reading http://starship.python.net/crew/theller/ctypes/ Reading http://sourceforge.net/project/showfiles.php?group_id=71702 Reading http://cheeseshop.python.org/pypi/ctypes/1.0.2 Best match: ctypes 1.0.2 Downloading http://downloads.sourceforge.net/ctypes/ctypes-1.0.2.win32-py2.4.exe?modtime=1179259132&big_mirror=0 Processing ctypes-1.0.2.win32-py2.4.exe ... creating c:\...\lib\site-packages\ctypes-1.0.2-py2.4-win32.egg Extracting ctypes-1.0.2-py2.4-win32.egg to c:\...\lib\site-packages Removing ctypes 0.9.6 from easy-install.pth file Adding ctypes 1.0.2 to easy-install.pth file Installed c:\...\lib\site-packages\ctypes-1.0.2-py2.4-win32.egg Processing dependencies for ctypes Finished processing dependencies for ctypes Linux: [root at sparrow ~]# easy_install-2.3 ctypes Processing ctypes Running setup.py -q bdist_egg --dist-dir /root/ctypes/egg-dist-tmp-OrObNN warning: no files found matching 'NEWS.txt' warning: no files found matching 'ctypes/.CTYPES_DEVEL' Configuring static FFI library: cd build/temp.linux-i686-2.3/libffi && env CFLAGS='' '/root/ctypes/source/libffi/configure' checking build system type... i686-pc-linux-gnu ... zip_safe flag not set; analyzing archive contents... ... Adding ctypes 1.0.1 to easy-install.pth file Installed /usr/lib/python2.3/site-packages/ctypes-1.0.1-py2.3-linux-i686.egg Processing dependencies for ctypes==1.0.1 From mhammond at skippinet.com.au Mon May 21 04:42:37 2007 From: mhammond at skippinet.com.au (Mark Hammond) Date: Mon, 21 May 2007 12:42:37 +1000 Subject: [Distutils] Adventures with x64, VS7 and VS8 on Windows Message-ID: <009f01c79b51$b1da52c0$1f0a0a0a@enfoldsystems.local> Hi all, I hope the cross-post is appropriate. I've started playing with getting the pywin32 extensions building under the AMD64 architecture. I started building with Visual Studio 8 (it was what I had handy) and I struck a few issues relating to the compiler version that I thought worth sharing. * In trying to build x64 from a 32bit VS7 (ie, cross-compiling via the PCBuild directory), the python.exe project fails with: pythoncore fatal error LNK1112: module machine type 'X86' conflicts with target machine type 'AMD64' is this a known issue, or am I doing something wrong? * The PCBuild8 project files appear to work without modification (I only tried native compilation here though, not a cross-compile) - however, unlike the PCBuild directory, they place all binaries in a 'PCBuild8/x64' directory. While this means that its possible to build for multiple architectures from the same source tree, it makes life harder for tools like 'distutils' - eg, distutils already knows about the 'PCBuild' directory, but it knows nothing about either PCBuild8 or PCBuild8/x64. A number of other build processes also know to look inside a PCBuild directory (eg, Mozilla), so instead of formalizing PCBuild8, I think we should merge PCBuild8 into PCBuild. This could mean PCBuild/vs7 and PCBuild/vs8 directories with the "project" files, but binaries would still be generated in the 'PCBuild' (or PCBuild/x64) directory. This would mean the same tree isn't capable of hosting 2 builds from different VS compilers, but I think that is reasonable (if it's a problem, just have multiple source directories). I understand that PCBuild8 is not "official", but in the assumption that future versions of Python will use a compiler later than VS7, it makes sense to me to clean this up now - what are others opinions on this? * Re the x64 directory used by the PCBuild8 process. IMO, it makes sense to generate x64 binaries to their own directory - my expectation is that cross-compiling between platforms is a reasonable use-case, and we should support multiple achitectures for the same compiler version. This would mean formalizing the x64 directory in both 'PCBuild' and distutils, and leaving other external build processes to update as they support x64 builds. Does this make sense? Would this fatally break other scripts used for packaging (eg, the MSI framework)? * Wide characters in VS8: PC/pyconfig.h defines PY_UNICODE_TYPE as 'unsigned short', which corresponds with both 'WCHAR' and 'wchar' in previous compiler versions. VS8 defines this as wchar_t, which I'm struggling to find a formal definition for beyond being 2 bytes. My problem is that code which assumes a 'Py_UNICODE *' could be used in place of a 'WCHAR *' now fails. I believe the intent on Windows has always been "PyUNICODE == 'native unicode'" - should PC/pyconfig.h reflect this (ie, should pyconfig.h grow a version specific definition of PyUNICODE as wchar_t)? * Finally, as something from left-field which may well take 12 months or more to pull off - but would there be any interest to moving the Windows build process to a cygwin environment based on the existing autoconf scripts? I know a couple of projects are doing this successfully, including Mozilla, so it has precendent. It does impose a greater burden on people trying to build on Windows, but I'd suggest that in recent times, many people who are likely to want to build Python on Windows are already likely to have a cygwin environment. Simpler mingw builds and nuking MSVC specific build stuff are among the advantages this would bring. It is not worth adding this as "yet another windows build option" - so IMO it is only worth progressing with if it became the "blessed" build process for windows - if there is support for this, I'll work on it as the opportunity presents itself... I'm (obviously) only suggesting we do this on the trunk and am happy to make all agreed changes - but I welcome all suggestions or critisisms of this endeavour... Cheers, Mark From martin at v.loewis.de Mon May 21 07:15:53 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 21 May 2007 07:15:53 +0200 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <009f01c79b51$b1da52c0$1f0a0a0a@enfoldsystems.local> References: <009f01c79b51$b1da52c0$1f0a0a0a@enfoldsystems.local> Message-ID: <46512B09.1080304@v.loewis.de> > * In trying to build x64 from a 32bit VS7 (ie, cross-compiling via the > PCBuild directory), the python.exe project fails with: > > pythoncore fatal error LNK1112: module machine type 'X86' conflicts with > target machine type 'AMD64' > > is this a known issue, or am I doing something wrong? You are likely doing something wrong: a) I assume it's VS 7.1 (i.e. VS.NET 2003); VS 2002 is not supported at all b) you probably didn't install vsextcomp, but you should. In fact, you don't need all of it, but you do need the cl.exe and link.exe wrappers it comes with - they dispatch to the proper tools from the SDK c) in case it isn't clear: you also need an AMD64 compiler, e.g. from the platform SDK. Unfortunately, Microsoft keeps changing the registry settings for the SDK, so vsextcomp only knows about some selected SDKs. If that causes a problem, please let me know. > * The PCBuild8 project files appear to work without modification (I only > tried native compilation here though, not a cross-compile) - however, unlike > the PCBuild directory, they place all binaries in a 'PCBuild8/x64' > directory. While this means that its possible to build for multiple > architectures from the same source tree, it makes life harder for tools like > 'distutils' - eg, distutils already knows about the 'PCBuild' directory, but > it knows nothing about either PCBuild8 or PCBuild8/x64. This is an issue to be discussed for Python 2.6. I'm personally hesitant to have the "official" build infrastructure deviate from the layout that has been in-use for so many years, as a lot of things depend on it. I don't find the need to have separate object directories convincing: For building the Win32/Win64 binaries, I have separate checkouts *anyway*, since all the add-on libraries would have to support multi-arch builds, but I think they don't. > A number of other build processes also know to look inside a PCBuild > directory (eg, Mozilla), so instead of formalizing PCBuild8, I think we > should merge PCBuild8 into PCBuild. Right - PCbuild8 should not get formalized. It probably should continue to be maintained. For 2.6, the first question to answer is: what compiler should it use? I would personally like to see Python "skip" VS 2005 altogether, as it will be soon superceded by Orcas. Unfortunately, it's unclear how long Microsoft will need to release Orcas (and also, when Python 2.6 will be released), so I would like to defer that question by a few months. > I understand that PCBuild8 is not "official", but in the > assumption that future versions of Python will use a compiler later than > VS7, it makes sense to me to clean this up now - what are others opinions on > this? Not "official" really only means "not used to build the official binaries" - just like PC/VC6. It's still (somewhat) maintained. As for cleaning it up - see above. I would *really* like to skip VS 2005 altogether, as I expect that soon after we decide to use VS 2005, Microsoft will replace it with the next release, stop supporting VS 2005, take the free compiler off the next, and so on (just like they did for VS 2003, soon after we decided to use it for 2.5). > * Re the x64 directory used by the PCBuild8 process. IMO, it makes sense to > generate x64 binaries to their own directory - my expectation is that > cross-compiling between platforms is a reasonable use-case, and we should > support multiple achitectures for the same compiler version. See above; I disagree. First, "multiple architectures" only means x86, AMD64, and Itanium, and I would like to drop "official" Itanium binaries from 2.6 (even though they could continue to be supported in the build process). Then, even if the Python build itself support multiple simultaneous architectures, the extension modules don't all (correct me if I'm wrong). > This would > mean formalizing the x64 directory in both 'PCBuild' and distutils, and > leaving other external build processes to update as they support x64 builds. > Does this make sense? Would this fatally break other scripts used for > packaging (eg, the MSI framework)? The MSI packaging would need to be changed, certainly. It currently detects the architecture it needs to package by looking at the file type of python.exe; that would have to be changed to give it an explicit parameter what architecture to package, or have it package all architectures it can find. > * Wide characters in VS8: PC/pyconfig.h defines PY_UNICODE_TYPE as 'unsigned > short', which corresponds with both 'WCHAR' and 'wchar' in previous compiler > versions. VS8 defines this as wchar_t, which I'm struggling to find a > formal definition for beyond being 2 bytes. In C or in C++? In C++, wchar_t is a builtin type, just like short, int, long. So there is no further formal definition. In C (including C99), wchar_t ought to be defined in stddef.h. > My problem is that code which > assumes a 'Py_UNICODE *' could be used in place of a 'WCHAR *' now fails. I > believe the intent on Windows has always been "PyUNICODE == 'native > unicode'" - should PC/pyconfig.h reflect this (ie, should pyconfig.h grow a > version specific definition of PyUNICODE as wchar_t)? I'd rather make it a platform-specific definition (for platform=Windows API). Correct me if I'm wrong, but isn't wchar_t also available in VS 2003 (and even in VC6?). And doesn't it have the "right" definition in all these compilers? So +1 for setting Py_UNICODE to wchar_t on Windows. > * Finally, as something from left-field which may well take 12 months or > more to pull off - but would there be any interest to moving the Windows > build process to a cygwin environment based on the existing autoconf > scripts? What compiler would you use there? I very much like using the VS debugger when developing on Windows, so that capability should not go away. Regards, Martin From peter.mowry at amd.com Mon May 21 08:11:10 2007 From: peter.mowry at amd.com (Mowry, Peter) Date: Mon, 21 May 2007 01:11:10 -0500 Subject: [Distutils] Compile Windows64 AMD64 In-Reply-To: <002301c79ae1$7b623020$1f0a0a0a@enfoldsystems.local> References: <475F4CA05E4E1B46A7213674E70EFA620241E069@SAUSEXMB2.amd.com> <002301c79ae1$7b623020$1f0a0a0a@enfoldsystems.local> Message-ID: <475F4CA05E4E1B46A7213674E70EFA620241E16D@SAUSEXMB2.amd.com> Thanks for the responses :-). Sounds like ctypes is a cleaner more generic extension module of what I was considering to develop manually. -----Original Message----- From: Mark Hammond [mailto:mhammond at skippinet.com.au] Sent: Sunday, May 20, 2007 8:19 AM To: Mowry, Peter; distutils-sig at python.org Subject: RE: [Distutils] Compile Windows64 AMD64 > So I tried ctypes, and it just worked (without any build issues - in fact I didn't > have to build anything or write any code). Is there any reason I'd want to write > an extend python module (to load my shared library and run exported functions from >it) that compiles with distutils, instead of just using ctypes? If your extension module does exactly what you describe - ie, load functions from a DLL and calls them, then ctypes is a perfect replacement. Mark From mhammond at skippinet.com.au Mon May 21 08:13:43 2007 From: mhammond at skippinet.com.au (Mark Hammond) Date: Mon, 21 May 2007 16:13:43 +1000 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <46512B09.1080304@v.loewis.de> Message-ID: <00af01c79b6f$2f863c30$1f0a0a0a@enfoldsystems.local> Hi Martin, > You are likely doing something wrong: > a) I assume it's VS 7.1 (i.e. VS.NET 2003); VS 2002 is not supported > at all > b) you probably didn't install vsextcomp, but you should. > In fact, you don't need all of it, but you do need the cl.exe and > link.exe wrappers it comes with - they dispatch to the proper > tools from the SDK > c) in case it isn't clear: you also need an AMD64 compiler, e.g. > from the platform SDK. > Unfortunately, Microsoft keeps changing the registry settings for > the SDK, so vsextcomp only knows about some selected SDKs. If > that causes a problem, please let me know. I'm using the full-blown VS.NET 2003, as given to a number of python-dev people by Microsoft a number of years ago. This appears to come with the SDK and a 64bit compiler. I'm guessing vsextcomp doesn't use the Visual Studio 'ReleaseAMD64' configuration - would it be OK for me to check in changes to the PCBuild projects for this configuration? > This is an issue to be discussed for Python 2.6. I'm > personally hesitant > to have the "official" build infrastructure deviate from the > layout that > has been in-use for so many years, as a lot of things depend on it. Yes, I agree - although I consider x64 new enough that an opportunity exists to set a new 'standard'. However, if most 'external' build processes will not otherwise need to change for a 64bit environment, then I agree that nothing should change in Python's layout. > Right - PCbuild8 should not get formalized. It probably > should continue to be maintained. So is there something we can do to make distutils play better with binaries built from PCBuild8, even though it is considered temporary? It seems the best thing might be to modify the PCBuild8 build process so the output binaries are in the ../PCBuild' directory - this way distutils and others continue to work fine. Does that sound reasonable? > For 2.6, the first question to answer is: what compiler should it use? > > I would personally like to see Python "skip" VS 2005 altogether, > as it will be soon superceded by Orcas. Unfortunately, it's unclear > how long Microsoft will need to release Orcas (and also, when Python > 2.6 will be released), so I would like to defer that question by > a few months. I've no objection to that - but I'd like to help keep the pain to a minimum for people who find themselves trying to build 64bit extensions in the meantime. Anecdotally, VS8 is the compiler most people start trying to use for this (quite possibly because that is what they already have handy). > See above; I disagree. First, "multiple architectures" only means x86, > AMD64, and Itanium, and I would like to drop "official" > Itanium binaries > from 2.6 (even though they could continue to be supported in the build > process). Then, even if the Python build itself support multiple > simultaneous architectures, the extension modules don't all (correct > me if I'm wrong). Yes, I agree that it is unlikely to work in practice - at least for a number of years as the external libs and extensions catch up. > > * Wide characters in VS8: PC/pyconfig.h defines > PY_UNICODE_TYPE as 'unsigned > > short', which corresponds with both 'WCHAR' and 'wchar' in > previous compiler > > versions. VS8 defines this as wchar_t, which I'm > struggling to find a > > formal definition for beyond being 2 bytes. > > In C or in C++? In C++, wchar_t is a builtin type, just like > short, int, > long. So there is no further formal definition. This was in C++, but the problem was really WCHAR, as used by much of the win32 API. > I'd rather make it a platform-specific definition (for > platform=Windows > API). Correct me if I'm wrong, but isn't wchar_t also available in VS > 2003 (and even in VC6?). And doesn't it have the "right" definition in > all these compilers? hrm - as above, I'm more concerned with the definition of WCHAR - which means my problem is related more to the Platform SDK version rather than the compiler. This is unfortunate - on one hand we do consider 'platform=Windows API', and WCHAR is very much an API concept. I'll need to dig some more into this, but at least I know I'm not wasting my time :) > > * Finally, as something from left-field which may well take > > 12 months or > > more to pull off - but would there be any interest to > > moving the Windows > > build process to a cygwin environment based on the existing autoconf > > scripts? > > What compiler would you use there? I very much like using the VS > debugger when developing on Windows, so that capability should not > go away. You would use whatever compiler the autoconf toolset found. Recent versions know enough about MSVC for simple projects. Many people would need to take care that their environment pointed at the correct compiler - especially the person building releases. But assuming MSVC was found and had the appropriate switches passed, there would be no impact on the ability to use Visual Studio as a debugging environment. Thanks, Mark From mal at egenix.com Mon May 21 13:28:04 2007 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 21 May 2007 13:28:04 +0200 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <4E9372E6B2234D4F859320D896059A9508DCBEE4B0@exchis.ccp.ad.local> References: <009f01c79b51$b1da52c0$1f0a0a0a@enfoldsystems.local> <46512B09.1080304@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE4B0@exchis.ccp.ad.local> Message-ID: <46518244.9050207@egenix.com> On 2007-05-21 12:30, Kristj?n Valur J?nsson wrote: >> >> [Py_UNICODE being #defined as "unsigned short" on Windows] >> >> I'd rather make it a platform-specific definition (for platform=Windows >> API). Correct me if I'm wrong, but isn't wchar_t also available in VS >> 2003 (and even in VC6?). And doesn't it have the "right" definition in >> all these compilers? >> >> So +1 for setting Py_UNICODE to wchar_t on Windows. > > Yes. Btw, in previous visual studio versions, wchar_t was not treated > as a builtin type by default, but rather as synonymous with unsighed short. > Now the default is that it is, and this causes some semantic differences > and incompatibilities of the type seen. +1 from me. If think this is simply a bug introduced with the UCS4 patches in Python 2.2. unicodeobject.h already has this code: #ifndef PY_UNICODE_TYPE /* Windows has a usable wchar_t type (unless we're using UCS-4) */ # if defined(MS_WIN32) && Py_UNICODE_SIZE == 2 # define HAVE_USABLE_WCHAR_T # define PY_UNICODE_TYPE wchar_t # endif # if defined(Py_UNICODE_WIDE) # define PY_UNICODE_TYPE Py_UCS4 # endif #endif But for some reason, pyconfig.h defines: /* Define as the integral type used for Unicode representation. */ #define PY_UNICODE_TYPE unsigned short /* Define as the size of the unicode type. */ #define Py_UNICODE_SIZE SIZEOF_SHORT /* Define if you have a useable wchar_t type defined in wchar.h; useable means wchar_t must be 16-bit unsigned type. (see Include/unicodeobject.h). */ #if Py_UNICODE_SIZE == 2 #define HAVE_USABLE_WCHAR_T #endif disabling the default settings in the unicodeobject.h. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 21 2007) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From kristjan at ccpgames.com Mon May 21 12:39:05 2007 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=) Date: Mon, 21 May 2007 10:39:05 +0000 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <00af01c79b6f$2f863c30$1f0a0a0a@enfoldsystems.local> References: <46512B09.1080304@v.loewis.de> <00af01c79b6f$2f863c30$1f0a0a0a@enfoldsystems.local> Message-ID: <4E9372E6B2234D4F859320D896059A9508DCBEE4BF@exchis.ccp.ad.local> > -----Original Message----- > This was in C++, but the problem was really WCHAR, as used by much of > the > win32 API. > > > I'd rather make it a platform-specific definition (for > > platform=Windows > > API). Correct me if I'm wrong, but isn't wchar_t also available in VS > > 2003 (and even in VC6?). And doesn't it have the "right" definition > in > > all these compilers? > > hrm - as above, I'm more concerned with the definition of WCHAR - which > means my problem is related more to the Platform SDK version rather > than the > compiler. This is unfortunate - on one hand we do consider > 'platform=Windows API', and WCHAR is very much an API concept. I'll > need to > dig some more into this, but at least I know I'm not wasting my time :) Mark, your problem may be related to a setting in the "c/c++ -> language" tab in the settings, where "treat wchar_t as a builtin type" default has changed. I recommend that we do treat it as a builtin, but the VS2003 default was "no" and the 2005 is "yes". Could this be contributing to your problem? Kristjan From kristjan at ccpgames.com Mon May 21 12:30:24 2007 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=) Date: Mon, 21 May 2007 10:30:24 +0000 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <46512B09.1080304@v.loewis.de> References: <009f01c79b51$b1da52c0$1f0a0a0a@enfoldsystems.local> <46512B09.1080304@v.loewis.de> Message-ID: <4E9372E6B2234D4F859320D896059A9508DCBEE4B0@exchis.ccp.ad.local> First of all, I have put some work into pcbuild8 recently and it works well. I am trying to drum up momentum for work on PCBuild8 next europython. See http://wiki.python.org/moin/EuroPython2007Sprints > -----Original Message----- > From: python-dev-bounces+kristjan=ccpgames.com at python.org > > I don't find the need to have separate object directories convincing: > For building the Win32/Win64 binaries, I have separate checkouts > *anyway*, since all the add-on libraries would have to support > multi-arch builds, but I think they don't. No they don't, but that doesn't mean that you need different checkouts for python, only the others. Anyway, this is indeed something I'd like to see addressed. I don't think we should ditch cross-compilation. It should simplify a lot of stuff, including buildbot setup and so on (if we get the buildbot infrastructure to support it). It is also very cumbersome, if you are working on a project, to have the binaries all end up in the same place. Doing interactive work on python, I frequently compile both the 32 bit and 64 bit versions for testing and it would be downright silly to have to rebuild everything every time. > I would personally like to see Python "skip" VS 2005 altogether, > as it will be soon superceded by Orcas. Unfortunately, it's unclear > how long Microsoft will need to release Orcas (and also, when Python > 2.6 will be released), so I would like to defer that question by > a few months. I think this is a bit unrealistic. Here we are in the middle of 2007, VS2005 has just got SP1, and VS2003 is still the "official" compiler. PCBuild8 is ready, it just needs a little bit of extra love and buildbots to make us able to release PGO versions of x86 and x64. Given the delay for getting even this far, waiting for Orcas and then someone to create PCBuild9, and then getting it up and running and so on will mean waiting another two years. > The MSI packaging would need to be changed, certainly. It currently > detects the architecture it needs to package by looking at the file > type of python.exe; that would have to be changed to give it an > explicit parameter what architecture to package, or have it package > all architectures it can find. I am not familiar with the msi packaging process at all. But here is something we should start to consider: VISTA support. This could mean some of: 1) supplying python.dll as a Side By Side assembly 2) Changing python install locations 3) Supporting shadow libraries, where .pyc files end up in a different hierarchy from the .py files. (useful for many things beside VISTA) 4) Signing the python dlls and executables 5) Providing user level manifests. Vista adoption is going very fast. We see 10% of our users have moved to vista and rising. > I'd rather make it a platform-specific definition (for platform=Windows > API). Correct me if I'm wrong, but isn't wchar_t also available in VS > 2003 (and even in VC6?). And doesn't it have the "right" definition in > all these compilers? > > So +1 for setting Py_UNICODE to wchar_t on Windows. Yes. Btw, in previous visual studio versions, wchar_t was not treated as a builtin type by default, but rather as synonymous with unsighed short. Now the default is that it is, and this causes some semantic differences and incompatibilities of the type seen. Kristjan From mhammond at skippinet.com.au Mon May 21 13:46:53 2007 From: mhammond at skippinet.com.au (Mark Hammond) Date: Mon, 21 May 2007 21:46:53 +1000 Subject: [Distutils] [Python-Dev] wchar_t (was Adventures with x64, VS7 and VS8) on Windows In-Reply-To: <4E9372E6B2234D4F859320D896059A9508DCBEE4BF@exchis.ccp.ad.local> Message-ID: <00d301c79b9d$baae8a00$1f0a0a0a@enfoldsystems.local> Kristj?n Valur J?nsson quoting me: > > hrm - as above, I'm more concerned with the definition of > > WCHAR - which > > means my problem is related more to the Platform SDK version rather > > than the > > compiler. This is unfortunate - on one hand we do consider > > 'platform=Windows API', and WCHAR is very much an API concept. I'll > > need to > > dig some more into this, but at least I know I'm not > > wasting my time :) > > Mark, your problem may be related to a setting in the "c/c++ > -> language" > tab in the settings, where "treat wchar_t as a builtin type" > default has > changed. I recommend that we do treat it as a builtin, but the VS2003 > default was "no" and the 2005 is "yes". Could this be contributing to > your problem? Thanks for the suggestion and for introducing me to that option - but it made no difference. I'm guessing its related to C++ - code such as the following: static PyObject *TestIBuild() { // obviously nonsense - the point is to test if it compiles. WCHAR *wval = PyUnicode_AS_UNICODE(Py_None); return PyUnicode_FromUnicode(wval, wcslen(wval)); } works everywhere - except in a pywin32 .cpp file built on x64:) That code results in: win32/src/win32apimodule.cpp(81) : error C2440: 'initializing' : cannot convert from 'Py_UNICODE *' to 'WCHAR *' Types pointed to are unrelated; conversion requires reinterpret_cast, C-style cast or function-style cast win32/src/win32apimodule.cpp(82) : error C2664: 'PyUnicodeUCS2_FromUnicode' : cannot convert parameter 1 from 'WCHAR *' to 'const Py_UNICODE *' Types pointed to are unrelated; conversion requires reinterpret_cast, C-style cast or function-style cast Unfortunately I don't have pywin32 building under vc8 on x32, but expect it to happen there too. pywin32 uses distutils, but I inspected the options passed and can't find anything to make a difference. /Zc:wchar_t and/or /Zc:wchar_t- seem to be the command-line settings for this flag and it also makes no difference. I'm out of time to confirm is is simply "c++ with vs8", but did confirm that the patch below appears to solve the problem, and given Martin's previous +1, I decided to stop there. I failed in a quick attempt at replacing the literal 2 with something involving sizeof. Does this look reasonable? Cheers, Mark Index: pyconfig.h =================================================================== --- pyconfig.h (revision 55487) +++ pyconfig.h (working copy) @@ -492,10 +492,10 @@ #define Py_USING_UNICODE /* Define as the integral type used for Unicode representation. */ -#define PY_UNICODE_TYPE unsigned short +#define PY_UNICODE_TYPE wchar_t /* Define as the size of the unicode type. */ -#define Py_UNICODE_SIZE SIZEOF_SHORT +#define Py_UNICODE_SIZE 2 /* Define if you have a useable wchar_t type defined in wchar.h; useable means wchar_t must be 16-bit unsigned type. (see -------------- next part -------------- A non-text attachment was scrubbed... Name: winmail.dat Type: application/ms-tnef Size: 3260 bytes Desc: not available Url : http://mail.python.org/pipermail/distutils-sig/attachments/20070521/12f28bf0/attachment.bin From kristjan at ccpgames.com Mon May 21 13:56:24 2007 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=) Date: Mon, 21 May 2007 11:56:24 +0000 Subject: [Distutils] [Python-Dev] wchar_t (was Adventures with x64, VS7 and VS8) on Windows In-Reply-To: <00d301c79b9d$baae8a00$1f0a0a0a@enfoldsystems.local> References: <4E9372E6B2234D4F859320D896059A9508DCBEE4BF@exchis.ccp.ad.local> <00d301c79b9d$baae8a00$1f0a0a0a@enfoldsystems.local> Message-ID: <4E9372E6B2234D4F859320D896059A9508DCBEE502@exchis.ccp.ad.local> > -----Original Message----- > below appears to solve the problem, and given Martin's previous +1, I > decided to stop there. I failed in a quick attempt at replacing the > literal 2 with something involving sizeof. Does this look reasonable? > +1. I should add that we have this local mod in our own patched python. It does seem like a mistake that needs fixing. Kristjan From mhammond at skippinet.com.au Mon May 21 14:33:27 2007 From: mhammond at skippinet.com.au (Mark Hammond) Date: Mon, 21 May 2007 22:33:27 +1000 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <4E9372E6B2234D4F859320D896059A9508DCBEE4B0@exchis.ccp.ad.local> Message-ID: <00e501c79ba4$3b9fd1e0$1f0a0a0a@enfoldsystems.local> Hi Kristj?n, > First of all, I have put some work into pcbuild8 recently and it works > well. It does! There are a few issues though, notably with distutils (and as mentioned before, any other tools what may assume PCBuild - see below) You quoting Martin: > > I don't find the need to have separate object directories > > convincing: > > For building the Win32/Win64 binaries, I have separate checkouts > > *anyway*, since all the add-on libraries would have to support > > multi-arch builds, but I think they don't. > > No they don't, but that doesn't mean that you need different checkouts > for python, only the others. Anyway, this is indeed > something I'd like to see addressed. I don't think we should ditch > cross-compilation. While I agree with you, Martin's point about the dependant libraries and tools is valid, and may defeat this goal. In the short term, we should research how some of these other projects are approaching x64 - we may also find that this helps with any autoconf work we choose to do. Ultimately, the person releasing the official binaries gets to say how this works (based on how they actually get it to work) > > I would personally like to see Python "skip" VS 2005 altogether, > > as it will be soon superceded by Orcas. Unfortunately, it's unclear > > how long Microsoft will need to release Orcas (and also, when Python > > 2.6 will be released), so I would like to defer that question by > > a few months. > I think this is a bit unrealistic. Here we are in the middle of 2007, > VS2005 has just got SP1, and VS2003 is still the "official" compiler. > PCBuild8 is ready, it just needs a little bit of extra love and > buildbots to make us able to release PGO versions of x86 and x64. > > Given the delay for getting even this far, waiting for Orcas and then > someone to create PCBuild9, and then getting it up and > running and so on > will mean waiting another two years. I don't believe there was any suggestion that Python 2.6 would wait for a compiler release from Microsoft. Before we talk about Vista and while I have your attention , some final questions relating to PCBuild8. Regardless of the ultimate layout for x64, what do you think about having PCBuid8 put the binaries into the PCBuild directory, and thus (theoretically) letting such a directory work with distutils and otherwise as a fully functional Python installation? > I am not familiar with the msi packaging process at all. But here is > something we should start to consider: VISTA support. This > could mean > some of: > 1) supplying python.dll as a Side By Side assembly Yes, this is something pywin32 is going to face. The hack of copying python*.dll into the 'system' directory - necessary for COM - is (sensibly!) no longer working. One thing at a time though... > 2) Changing python install locations > 3) Supporting shadow libraries, where .pyc files end up in a different > hierarchy from the .py files. (useful for many things > beside VISTA) > 4) Signing the python dlls and executables > 5) Providing user level manifests. And dragging distutils back op topic, having bdist_wininst supply a manifest that indicates escalation is required appears necessary. > Vista adoption is going very fast. We see 10% of our users have moved > to vista and rising. ack - I'm yet to try a 32 bit version, but my Vista-x64 box isn't proving very reliable yet. It *is* very pretty and cute though :) I'm surprised to see 10% unless your users are skewed towards early-adopters though, but I'm in no position to refute it! Cheers, Mark -------------- next part -------------- A non-text attachment was scrubbed... Name: winmail.dat Type: application/ms-tnef Size: 3744 bytes Desc: not available Url : http://mail.python.org/pipermail/distutils-sig/attachments/20070521/d9e6d8c2/attachment-0001.bin From jim at zope.com Mon May 21 22:50:19 2007 From: jim at zope.com (Jim Fulton) Date: Mon, 21 May 2007 16:50:19 -0400 Subject: [Distutils] setuptools process_url doesn't handle URLErrors gracefullly In-Reply-To: <1EF5ED10-6ADD-4355-9B06-04C9E3B69150@zope.com> References: <1178871702.22778.8.camel@mindy.whq.gocept.com> <20070515174827.ECD2E3A4036@sparrow.telecommunity.com> <1EF5ED10-6ADD-4355-9B06-04C9E3B69150@zope.com> Message-ID: On May 18, 2007, at 12:51 PM, Jim Fulton wrote: > > On May 15, 2007, at 1:50 PM, Phillip J. Eby wrote: > >> At 09:52 AM 5/14/2007 -0400, Jim Fulton wrote: >>> This problem is rather serious. As we rely more and more on >>> setuptools, it needs to be robust in the presence of this sort of >>> failure. >>> >>> Phillip, If I fix this, can we get a timely release with the fix? >> >> Is sometime next week sufficient to be described as timely? If so, >> then yes. > > Yes, great, sort of. Now I have to find time to actually make the > fix. I'll try. :) Actually, you already fixed it back in January. So any time you want to make that release ... :) Jim -- Jim Fulton mailto:jim at zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From martin at v.loewis.de Mon May 21 23:29:51 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 21 May 2007 23:29:51 +0200 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <00af01c79b6f$2f863c30$1f0a0a0a@enfoldsystems.local> References: <00af01c79b6f$2f863c30$1f0a0a0a@enfoldsystems.local> Message-ID: <46520F4F.5010502@v.loewis.de> > I'm using the full-blown VS.NET 2003, as given to a number of python-dev > people by Microsoft a number of years ago. This appears to come with the > SDK and a 64bit compiler. Not sure what it makes it appear to you that way - it doesn't. VS.NET 2003 is x86 only > I'm guessing vsextcomp doesn't use the Visual > Studio 'ReleaseAMD64' configuration - would it be OK for me to check in > changes to the PCBuild projects for this configuration? Please don't. It exclusively relies on vsextcomp, and is only useful if you have that infrastructure installed. See for yourself: it uses the /USE_CL:MS_OPTERON command line switch, which isn't a Microsoft invention (but instead invented by Peter Tr?ger). > So is there something we can do to make distutils play better with binaries > built from PCBuild8, even though it is considered temporary? In what way better? It already supports them just fine, AFAICT. I guess you are referring to the support for building extensions on top of a source tree "installation". I doubt that is used that often (but I understand you are using it). > It seems the > best thing might be to modify the PCBuild8 build process so the output > binaries are in the ../PCBuild' directory - this way distutils and others > continue to work fine. Does that sound reasonable? I think Kristjan will have to say a word here: I think he just likes it the way it is right now. That would rather suggest that build_ext needs to be changed. > I've no objection to that - but I'd like to help keep the pain to a minimum > for people who find themselves trying to build 64bit extensions in the > meantime. I recommend that those people install the official binaries. Why do you need to build the binaries from source, if all you want is to build extensions? >> In C or in C++? In C++, wchar_t is a builtin type, just like >> short, int, >> long. So there is no further formal definition. > > This was in C++, but the problem was really WCHAR, as used by much of the > win32 API. But in C, WCHAR should not be a problem (and I would like to see explicit source code and explicit compiler error message to be proven wrong). >>> * Finally, as something from left-field which may well take >>> 12 months or >>> more to pull off - but would there be any interest to >>> moving the Windows >>> build process to a cygwin environment based on the existing autoconf >>> scripts? >> What compiler would you use there? I very much like using the VS >> debugger when developing on Windows, so that capability should not >> go away. > > You would use whatever compiler the autoconf toolset found. Recent versions > know enough about MSVC for simple projects. Many people would need to take > care that their environment pointed at the correct compiler - especially the > person building releases. > > But assuming MSVC was found and had the appropriate switches passed, there > would be no impact on the ability to use Visual Studio as a debugging > environment. I doubt that could work in practice. You will have to rewrite everything to make it pass the correct command line switches. Regards, Martin From martin at v.loewis.de Mon May 21 23:43:39 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 21 May 2007 23:43:39 +0200 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <4E9372E6B2234D4F859320D896059A9508DCBEE4B0@exchis.ccp.ad.local> References: <009f01c79b51$b1da52c0$1f0a0a0a@enfoldsystems.local> <46512B09.1080304@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE4B0@exchis.ccp.ad.local> Message-ID: <4652128B.3000301@v.loewis.de> >> I don't find the need to have separate object directories convincing: >> For building the Win32/Win64 binaries, I have separate checkouts >> *anyway*, since all the add-on libraries would have to support >> multi-arch builds, but I think they don't. > > No they don't, but that doesn't mean that you need different checkouts > for python, only the others. Anyway, this is indeed something I'd like > to see addressed. I don't think we should ditch cross-compilation. Nobody proposed to ditch cross-compilation. I very much like cross-compilation, I do all Itanium and AMD64 in cross-compilation. I just want the *file structure* of the output to be the very same for all architectures, meaning that they can't coexist in a single source directory. > It > should simplify a lot of stuff, including buildbot setup and so on (if > we get the buildbot infrastructure to support it). It is also very > cumbersome, if you are working on a project, to have the binaries all > end up in the same place. Doing interactive work on python, I frequently > compile both the 32 bit and 64 bit versions for testing and it would > be downright silly to have to rebuild everything every time. No, you use two checkouts, of course. > I think this is a bit unrealistic. Here we are in the middle of 2007, > VS2005 has just got SP1, and VS2003 is still the "official" compiler. > PCBuild8 is ready, it just needs a little bit of extra love and > buildbots to make us able to release PGO versions of x86 and x64. No matter what the next compiler is (VS 2005 or VS 2007/2008), it's still *a lot* of work until the VS 2005 build can be used for releasing Python. For example, there is no support for the SxS installation of msvcr8.dll, using the MSI merge module. > Given the delay for getting even this far, waiting for Orcas and then > someone to create PCBuild9, and then getting it up and running and so on > will mean waiting another two years. No. I would expect that either the PCbuild or PCbuild8 project files can be used with little changes to build using VS9. I just tried, and it works reasonably well. > I am not familiar with the msi packaging process at all. But here is > something we should start to consider: VISTA support. This could mean > some of: Not sure whether anything really is needed. Python works fine on Vista. > 1) supplying python.dll as a Side By Side assembly What would that improve? > 2) Changing python install locations Why? > 3) Supporting shadow libraries, where .pyc files end up in a different > hierarchy from the .py files. (useful for many things beside VISTA) Yes, and people had written PEPs for it which failed due to lack of follow up. > 4) Signing the python dlls and executables Hmm. > 5) Providing user level manifests. > > Vista adoption is going very fast. We see 10% of our users have moved > to vista and rising. Sure, and have they reported problems with Python on Vista (problems specific to Vista?) > Yes. Btw, in previous visual studio versions, wchar_t was not treated > as a builtin type by default, but rather as synonymous with unsighed short. > Now the default is that it is, and this causes some semantic differences > and incompatibilities of the type seen. C or C++? According to the standards, it ought to be a builtin, primitive type in C++, and a typedef in C. Regards, Martin From mhammond at skippinet.com.au Tue May 22 04:58:45 2007 From: mhammond at skippinet.com.au (Mark Hammond) Date: Tue, 22 May 2007 12:58:45 +1000 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <46518244.9050207@egenix.com> Message-ID: <016c01c79c1d$1db9d0d0$1f0a0a0a@enfoldsystems.local> Hi Marc-Andre, > +1 from me. > > If think this is simply a bug introduced with the UCS4 patches in > Python 2.2. > > unicodeobject.h already has this code: > > #ifndef PY_UNICODE_TYPE > > /* Windows has a usable wchar_t type (unless we're using UCS-4) */ > # if defined(MS_WIN32) && Py_UNICODE_SIZE == 2 > # define HAVE_USABLE_WCHAR_T > # define PY_UNICODE_TYPE wchar_t > # endif > > # if defined(Py_UNICODE_WIDE) > # define PY_UNICODE_TYPE Py_UCS4 > # endif > > #endif > > But for some reason, pyconfig.h defines: > > /* Define as the integral type used for Unicode representation. */ > #define PY_UNICODE_TYPE unsigned short > > /* Define as the size of the unicode type. */ > #define Py_UNICODE_SIZE SIZEOF_SHORT > > /* Define if you have a useable wchar_t type defined in > wchar.h; useable > means wchar_t must be 16-bit unsigned type. (see > Include/unicodeobject.h). */ > #if Py_UNICODE_SIZE == 2 > #define HAVE_USABLE_WCHAR_T > #endif > > disabling the default settings in the unicodeobject.h. Yes, that does appear strange. The following patch works for me, keeps Python building and appears to solve my problem. Any objections? Mark Index: pyconfig.h =================================================================== --- pyconfig.h (revision 55487) +++ pyconfig.h (working copy) @@ -491,22 +491,13 @@ /* Define if you want to have a Unicode type. */ #define Py_USING_UNICODE -/* Define as the integral type used for Unicode representation. */ -#define PY_UNICODE_TYPE unsigned short - /* Define as the size of the unicode type. */ -#define Py_UNICODE_SIZE SIZEOF_SHORT +/* This is enough for unicodeobject.h to do the "right thing" on Windows. */ +#define Py_UNICODE_SIZE 2 -/* Define if you have a useable wchar_t type defined in wchar.h; useable - means wchar_t must be 16-bit unsigned type. (see - Include/unicodeobject.h). */ -#if Py_UNICODE_SIZE == 2 -#define HAVE_USABLE_WCHAR_T - /* Define to indicate that the Python Unicode representation can be passed as-is to Win32 Wide API. */ #define Py_WIN_WIDE_FILENAMES -#endif /* Use Python's own small-block memory-allocator. */ #define WITH_PYMALLOC 1 From mhammond at skippinet.com.au Tue May 22 05:25:35 2007 From: mhammond at skippinet.com.au (Mark Hammond) Date: Tue, 22 May 2007 13:25:35 +1000 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <46520F4F.5010502@v.loewis.de> Message-ID: <017001c79c20$dd347e80$1f0a0a0a@enfoldsystems.local> Hi Martin, > > I'm using the full-blown VS.NET 2003, as given to a number > of python-dev > > people by Microsoft a number of years ago. This appears to > come with the > > SDK and a 64bit compiler. > > Not sure what it makes it appear to you that way - it doesn't. VS.NET > 2003 is x86 only Yes - I was confused by finding an x64 configuration option, and this looking very similar to VC8. My apologies for the confusion. > > So is there something we can do to make distutils play > better with binaries > > built from PCBuild8, even though it is considered temporary? > > In what way better? It already supports them just fine, AFAICT. > > I guess you are referring to the support for building extensions on > top of a source tree "installation". I doubt that is used that often > (but I understand you are using it). Yes, that is correct. I agree it will be rarely used by Python user's, but believe it is a common scenario for people who maintain extensions or libraries, particularly those who want debugging builds. > > It seems the > > best thing might be to modify the PCBuild8 build process so the output > > binaries are in the ../PCBuild' directory - this way distutils and others > > continue to work fine. Does that sound reasonable? > I think Kristjan will have to say a word here: I think he just likes > it the way it is right now. That would rather suggest that build_ext > needs to be changed. So this means PCBuild8 does indeed get formalized into distutils? I'm happy to live with it if it lets me work, but it seems contrary to our emails yesterday. It would also mean the PCBuild8 environment will not work with external build processes that assume the standard layout, but that really isn't something I'm willing to run the pydev gauntlet for at the current time. > > I've no objection to that - but I'd like to help keep the > pain to a minimum > > for people who find themselves trying to build 64bit > extensions in the > > meantime. > > I recommend that those people install the official binaries. > Why do you > need to build the binaries from source, if all you want is to build > extensions? Let's assume that people have a valid reason to build from source - it really is quite common in the open source world. The meta-question then becomes "is it reasonable to expect people be able to build extensions from a tree built with VC8, in the same way they can with VS7?". I think you are suggesting we do want to support this, but I want to be clear. > > >> In C or in C++? In C++, wchar_t is a builtin type, just like > >> short, int, > >> long. So there is no further formal definition. > > > > This was in C++, but the problem was really WCHAR, as used > by much of the > > win32 API. > > But in C, WCHAR should not be a problem (and I would like to see > explicit source code and explicit compiler error message to be > proven wrong). Please see my other mail to Kristjan - at this stage I can only reproduce it in C++ on VC8. C++ on VC7 and C on VC8 both work fine. Please let me know if thde code snippet I pasted or the compiler error aren't suitable. I've stopped further investigation since there seems support for a change that makes the problem go away. > >>> * Finally, as something from left-field which may well take > >>> 12 months or > >>> more to pull off - but would there be any interest to > >>> moving the Windows > >>> build process to a cygwin environment based on the > existing autoconf > >>> scripts? > >> What compiler would you use there? I very much like using the VS > >> debugger when developing on Windows, so that capability should not > >> go away. > > > > You would use whatever compiler the autoconf toolset found. > Recent versions > > know enough about MSVC for simple projects. Many people > would need to take > > care that their environment pointed at the correct compiler > - especially the > > person building releases. > > > > But assuming MSVC was found and had the appropriate > switches passed, there > > would be no impact on the ability to use Visual Studio as a > debugging > > environment. > > I doubt that could work in practice. You will have to rewrite > everything > to make it pass the correct command line switches. 'have to rewrite everything' sounds a little pessimistic, but that's fine with me - consider this idea dropped. Cheers, Mark From steve at holdenweb.com Tue May 22 07:01:29 2007 From: steve at holdenweb.com (Steve Holden) Date: Tue, 22 May 2007 01:01:29 -0400 Subject: [Distutils] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <4E9372E6B2234D4F859320D896059A9508DCBEE4B0@exchis.ccp.ad.local> References: <009f01c79b51$b1da52c0$1f0a0a0a@enfoldsystems.local> <46512B09.1080304@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE4B0@exchis.ccp.ad.local> Message-ID: <46527929.8040600@holdenweb.com> Kristj?n Valur J?nsson wrote: > First of all, I have put some work into pcbuild8 recently and it works > well. I am trying to drum up momentum for work on PCBuild8 > next europython. See http://wiki.python.org/moin/EuroPython2007Sprints > > >> -----Original Message----- >> From: python-dev-bounces+kristjan=ccpgames.com at python.org >> >> I don't find the need to have separate object directories convincing: >> For building the Win32/Win64 binaries, I have separate checkouts >> *anyway*, since all the add-on libraries would have to support >> multi-arch builds, but I think they don't. > > No they don't, but that doesn't mean that you need different checkouts > for python, only the others. Anyway, this is indeed something I'd like > to see addressed. I don't think we should ditch cross-compilation. It > should simplify a lot of stuff, including buildbot setup and so on (if > we get the buildbot infrastructure to support it). It is also very > cumbersome, if you are working on a project, to have the binaries all > end up in the same place. Doing interactive work on python, I frequently > compile both the 32 bit and 64 bit versions for testing and it would > be downright silly to have to rebuild everything every time. > >> I would personally like to see Python "skip" VS 2005 altogether, >> as it will be soon superceded by Orcas. Unfortunately, it's unclear >> how long Microsoft will need to release Orcas (and also, when Python >> 2.6 will be released), so I would like to defer that question by >> a few months. > I think this is a bit unrealistic. Here we are in the middle of 2007, > VS2005 has just got SP1, and VS2003 is still the "official" compiler. > PCBuild8 is ready, it just needs a little bit of extra love and > buildbots to make us able to release PGO versions of x86 and x64. > > Given the delay for getting even this far, waiting for Orcas and then > someone to create PCBuild9, and then getting it up and running and so on > will mean waiting another two years. > Addressing only the issues of PCBuild8 and 64-bit architectures, I have tried to establish "free" buildbot support for the 64-bit architectures without any real success. Should the PSF be considering paying for infrastructure that will support 64-bit build reporting? > [...] regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC/Ltd http://www.holdenweb.com Skype: holdenweb http://del.icio.us/steve.holden ------------------ Asciimercial --------------------- Get on the web: Blog, lens and tag your way to fame!! holdenweb.blogspot.com squidoo.com/pythonology tagged items: del.icio.us/steve.holden/python All these services currently offer free registration! -------------- Thank You for Reading ---------------- From steve at holdenweb.com Tue May 22 07:01:29 2007 From: steve at holdenweb.com (Steve Holden) Date: Tue, 22 May 2007 01:01:29 -0400 Subject: [Distutils] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <4E9372E6B2234D4F859320D896059A9508DCBEE4B0@exchis.ccp.ad.local> References: <009f01c79b51$b1da52c0$1f0a0a0a@enfoldsystems.local> <46512B09.1080304@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE4B0@exchis.ccp.ad.local> Message-ID: <46527929.8040600@holdenweb.com> Kristj?n Valur J?nsson wrote: > First of all, I have put some work into pcbuild8 recently and it works > well. I am trying to drum up momentum for work on PCBuild8 > next europython. See http://wiki.python.org/moin/EuroPython2007Sprints > > >> -----Original Message----- >> From: python-dev-bounces+kristjan=ccpgames.com at python.org >> >> I don't find the need to have separate object directories convincing: >> For building the Win32/Win64 binaries, I have separate checkouts >> *anyway*, since all the add-on libraries would have to support >> multi-arch builds, but I think they don't. > > No they don't, but that doesn't mean that you need different checkouts > for python, only the others. Anyway, this is indeed something I'd like > to see addressed. I don't think we should ditch cross-compilation. It > should simplify a lot of stuff, including buildbot setup and so on (if > we get the buildbot infrastructure to support it). It is also very > cumbersome, if you are working on a project, to have the binaries all > end up in the same place. Doing interactive work on python, I frequently > compile both the 32 bit and 64 bit versions for testing and it would > be downright silly to have to rebuild everything every time. > >> I would personally like to see Python "skip" VS 2005 altogether, >> as it will be soon superceded by Orcas. Unfortunately, it's unclear >> how long Microsoft will need to release Orcas (and also, when Python >> 2.6 will be released), so I would like to defer that question by >> a few months. > I think this is a bit unrealistic. Here we are in the middle of 2007, > VS2005 has just got SP1, and VS2003 is still the "official" compiler. > PCBuild8 is ready, it just needs a little bit of extra love and > buildbots to make us able to release PGO versions of x86 and x64. > > Given the delay for getting even this far, waiting for Orcas and then > someone to create PCBuild9, and then getting it up and running and so on > will mean waiting another two years. > Addressing only the issues of PCBuild8 and 64-bit architectures, I have tried to establish "free" buildbot support for the 64-bit architectures without any real success. Should the PSF be considering paying for infrastructure that will support 64-bit build reporting? > [...] regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC/Ltd http://www.holdenweb.com Skype: holdenweb http://del.icio.us/steve.holden ------------------ Asciimercial --------------------- Get on the web: Blog, lens and tag your way to fame!! holdenweb.blogspot.com squidoo.com/pythonology tagged items: del.icio.us/steve.holden/python All these services currently offer free registration! -------------- Thank You for Reading ---------------- From martin at v.loewis.de Tue May 22 07:20:40 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 22 May 2007 07:20:40 +0200 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <017001c79c20$dd347e80$1f0a0a0a@enfoldsystems.local> References: <017001c79c20$dd347e80$1f0a0a0a@enfoldsystems.local> Message-ID: <46527DA8.1030701@v.loewis.de> > Yes, that is correct. I agree it will be rarely used by Python user's, but > believe it is a common scenario for people who maintain extensions or > libraries, particularly those who want debugging builds. Ah, debugging builds. It's true that PCbuild does not support them for AMD64, and it's also true that you need such a build if you want the debug CRT. However, I think people ask much too often for a debugging build; in many cases, they could work happily with a release build that supports debugging. Depending on the problem you try to solve, you may or may not need debug information for pythonxy.dll as well, or just for your extension module. I'd like to repeat an offer that I have made several times in the past: if somebody contributes a patch to msi.py which packs all PDB files (and whatever you else you need) into a ZIP file (or whatever else works) from the release build, then I could happily release PDB files along with the actual installer files. (I would not like to release the PDB files *in* the installer files, as I expect they would blow up the msi size significantly). >> I think Kristjan will have to say a word here: I think he just likes >> it the way it is right now. That would rather suggest that build_ext >> needs to be changed. > > So this means PCBuild8 does indeed get formalized into distutils? I'm happy > to live with it if it lets me work, but it seems contrary to our emails > yesterday. It would mean that - I'm willing to compromise to make Kristjan happy (he contributed PCbuild8, so he has to decide what changes to it are acceptable and which aren't). A middle ground might be an addition build step (e.g. as a .bat file) which copies all result files also into PCbuild. > Let's assume that people have a valid reason to build from source - it > really is quite common in the open source world. The meta-question then > becomes "is it reasonable to expect people be able to build extensions from > a tree built with VC8, in the same way they can with VS7?". I think you are > suggesting we do want to support this, but I want to be clear. As long as people are willing to maintain it - why not? It's not "official" in the sense that if it breaks, nobody might fix it, but it doesn't hurt to have it (AFAICT). Also, it not being official might mean that we are not obliged to provide backwards compatibility for it, so it can go away without notice (e.g. when/if PCbuild8 is dropped). The same could be done for PC/VC6, if anybody cared to contribute and maintain it. I think there are several other cases in distutils to support special cases (e.g. the DISTUTILS_USE_SDK environment variable); if people want to see their setup supported, AND ARE WILLING TO CONTRIBUTE: more power to them. > Please see my other mail to Kristjan - at this stage I can only reproduce it > in C++ on VC8. C++ on VC7 and C on VC8 both work fine. Please let me know > if thde code snippet I pasted or the compiler error aren't suitable. It's fine. I readily believe that it causes problems in C++. Regards, Martin From martin at v.loewis.de Tue May 22 07:32:42 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 22 May 2007 07:32:42 +0200 Subject: [Distutils] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <46527929.8040600@holdenweb.com> References: <009f01c79b51$b1da52c0$1f0a0a0a@enfoldsystems.local> <46512B09.1080304@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE4B0@exchis.ccp.ad.local> <46527929.8040600@holdenweb.com> Message-ID: <4652807A.2030707@v.loewis.de> > Addressing only the issues of PCBuild8 and 64-bit architectures, I > have tried to establish "free" buildbot support for the 64-bit > architectures without any real success. > > Should the PSF be considering paying for infrastructure that will > support 64-bit build reporting? You can bring it up to the board, of course. However, given that all other buildbot machines were contributed by volunteers, the fact that nobody volunteers to contribute buildbot machines for PCbuild8 indicates that there is not a lot of interest in that build infrastructure. Note that there *are* 64-bit buildbot slaves, e.g. for AMD64 (contributed by Neal Norwitz), Alpha (contributed by Neal Norwitz), Itanium (contributed by Matthias Klose, offering a machine from the Debian project), and PPC64 (likewise). These machines all run Linux or Tru64, and (to my understanding) serve other purposes as well, making the buildbot slave just a minor detail. It's an unfortunate fact of life that Microsoft Windows does not support multi-tasking multi-user workloads that well, so the Windows buildbot slave are (to my knowledge) typically dedicated machines (except for Tim's machine, which is just disconnected from the master when he doesn't feel like running buildbot jobs). Regards, Martin From titus at caltech.edu Tue May 22 07:52:57 2007 From: titus at caltech.edu (Titus Brown) Date: Mon, 21 May 2007 22:52:57 -0700 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <4652807A.2030707@v.loewis.de> References: <009f01c79b51$b1da52c0$1f0a0a0a@enfoldsystems.local> <46512B09.1080304@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE4B0@exchis.ccp.ad.local> <46527929.8040600@holdenweb.com> <4652807A.2030707@v.loewis.de> Message-ID: <20070522055257.GA13609@caltech.edu> On Tue, May 22, 2007 at 07:32:42AM +0200, "Martin v. L?wis" wrote: -> > Addressing only the issues of PCBuild8 and 64-bit architectures, I -> > have tried to establish "free" buildbot support for the 64-bit -> > architectures without any real success. -> > -> > Should the PSF be considering paying for infrastructure that will -> > support 64-bit build reporting? -> -> You can bring it up to the board, of course. However, given that -> all other buildbot machines were contributed by volunteers, the -> fact that nobody volunteers to contribute buildbot machines for -> PCbuild8 indicates that there is not a lot of interest in that -> build infrastructure. -> -> Note that there *are* 64-bit buildbot slaves, e.g. for AMD64 -> (contributed by Neal Norwitz), Alpha (contributed by Neal -> Norwitz), Itanium (contributed by Matthias Klose, offering -> a machine from the Debian project), and PPC64 (likewise). -> -> These machines all run Linux or Tru64, and (to my understanding) -> serve other purposes as well, making the buildbot slave -> just a minor detail. -> -> It's an unfortunate fact of life that Microsoft Windows does not -> support multi-tasking multi-user workloads that well, so the -> Windows buildbot slave are (to my knowledge) typically dedicated -> machines (except for Tim's machine, which is just disconnected -> from the master when he doesn't feel like running buildbot -> jobs). I haven't really been listening to this conversation, so forgive me if this isn't relevant, but: I'd be happy to install Windows and the latest VisualStudio on a 64-bit VMware image. I just can't be responsible for day-to-day administration of the buildslave; Windows requires too much attention for me to do that. cheers, --titus From mhammond at skippinet.com.au Tue May 22 08:29:36 2007 From: mhammond at skippinet.com.au (Mark Hammond) Date: Tue, 22 May 2007 16:29:36 +1000 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <46527DA8.1030701@v.loewis.de> Message-ID: <019101c79c3a$91d06b60$1f0a0a0a@enfoldsystems.local> > However, I think people ask much too often for a debugging build; > in many cases, they could work happily with a release build that > supports debugging. Depending on the problem you try to solve, you > may or may not need debug information for pythonxy.dll as well, > or just for your extension module. While that is true in theory, I often find it is not the case in practice, generally due to the optimizer. Depending on what magic the compiler has done, it can be very difficult to set breakpoints (conditional or otherwise), inspect variable values, etc. It is useful in some cases, but very often I find myself falling back to a debug build after attempting to debug a local release build. > I think there are several other cases in distutils to support > special cases (e.g. the DISTUTILS_USE_SDK environment variable); > if people want to see their setup supported, AND ARE WILLING TO > CONTRIBUTE: more power to them. Yes, but I believe its also important to solicit feedback on the best way to achieve their aims. In this particular case, I believe it would have been misguided for me to simply check in whatever was necessary to have distutils work from the PCBuild8 directory. I hope it is clear that I am willing to contribute the outcome of these discussions... Cheers, Mark From mal at egenix.com Tue May 22 10:57:22 2007 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 22 May 2007 10:57:22 +0200 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <016c01c79c1d$1db9d0d0$1f0a0a0a@enfoldsystems.local> References: <016c01c79c1d$1db9d0d0$1f0a0a0a@enfoldsystems.local> Message-ID: <4652B072.2040002@egenix.com> Hi Mark, >> +1 from me. >> >> I think this is simply a bug introduced with the UCS4 patches in >> Python 2.2. >> >> unicodeobject.h already has this code: >> >> #ifndef PY_UNICODE_TYPE >> >> /* Windows has a usable wchar_t type (unless we're using UCS-4) */ >> # if defined(MS_WIN32) && Py_UNICODE_SIZE == 2 >> # define HAVE_USABLE_WCHAR_T >> # define PY_UNICODE_TYPE wchar_t >> # endif >> >> # if defined(Py_UNICODE_WIDE) >> # define PY_UNICODE_TYPE Py_UCS4 >> # endif >> >> #endif >> >> But for some reason, pyconfig.h defines: >> >> /* Define as the integral type used for Unicode representation. */ >> #define PY_UNICODE_TYPE unsigned short >> >> /* Define as the size of the unicode type. */ >> #define Py_UNICODE_SIZE SIZEOF_SHORT >> >> /* Define if you have a useable wchar_t type defined in >> wchar.h; useable >> means wchar_t must be 16-bit unsigned type. (see >> Include/unicodeobject.h). */ >> #if Py_UNICODE_SIZE == 2 >> #define HAVE_USABLE_WCHAR_T >> #endif >> >> disabling the default settings in the unicodeobject.h. > > Yes, that does appear strange. The following patch works for me, keeps > Python building and appears to solve my problem. Any objections? Looks fine to me. > Mark > > > Index: pyconfig.h > =================================================================== > --- pyconfig.h (revision 55487) > +++ pyconfig.h (working copy) > @@ -491,22 +491,13 @@ > /* Define if you want to have a Unicode type. */ > #define Py_USING_UNICODE > > -/* Define as the integral type used for Unicode representation. */ > -#define PY_UNICODE_TYPE unsigned short > - > /* Define as the size of the unicode type. */ > -#define Py_UNICODE_SIZE SIZEOF_SHORT > +/* This is enough for unicodeobject.h to do the "right thing" on Windows. > */ > +#define Py_UNICODE_SIZE 2 > > -/* Define if you have a useable wchar_t type defined in wchar.h; useable > - means wchar_t must be 16-bit unsigned type. (see > - Include/unicodeobject.h). */ > -#if Py_UNICODE_SIZE == 2 > -#define HAVE_USABLE_WCHAR_T > - > /* Define to indicate that the Python Unicode representation can be passed > as-is to Win32 Wide API. */ > #define Py_WIN_WIDE_FILENAMES > -#endif > > /* Use Python's own small-block memory-allocator. */ > #define WITH_PYMALLOC 1 > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/mal%40egenix.com -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 22 2007) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From kristjan at ccpgames.com Tue May 22 12:02:17 2007 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=) Date: Tue, 22 May 2007 10:02:17 +0000 Subject: [Distutils] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <4652807A.2030707@v.loewis.de> References: <009f01c79b51$b1da52c0$1f0a0a0a@enfoldsystems.local> <46512B09.1080304@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE4B0@exchis.ccp.ad.local> <46527929.8040600@holdenweb.com> <4652807A.2030707@v.loewis.de> Message-ID: <4E9372E6B2234D4F859320D896059A9508DCBEE6D6@exchis.ccp.ad.local> > -----Original Message----- > From: "Martin v. L?wis" [mailto:martin at v.loewis.de] > Sent: Tuesday, May 22, 2007 05:33 > To: Steve Holden > Cc: Kristj?n Valur J?nsson; Mark Hammond; distutils-sig at python.org; > python-dev at python.org > Subject: Re: Adventures with x64, VS7 and VS8 on Windows > > > Addressing only the issues of PCBuild8 and 64-bit architectures, I > > have tried to establish "free" buildbot support for the 64-bit > > architectures without any real success. > > > > Should the PSF be considering paying for infrastructure that will > > support 64-bit build reporting? > > You can bring it up to the board, of course. However, given that > all other buildbot machines were contributed by volunteers, the > fact that nobody volunteers to contribute buildbot machines for > PCbuild8 indicates that there is not a lot of interest in that > build infrastructure. Most people just install whatever it is that they are offered to download. For me, the most compelling reason to provide the new builds is the 15% performance benefit. If there are no technical and corporate difficulties, such as firewalls and security, I am sure that CCP can provide a VisualStudio2005 buildbot for our needs. Wasn't there some issue that each buildbot can only provide a single build? Here is a place where cross-compilation comes into its own again. Kristjan From kristjan at ccpgames.com Tue May 22 12:31:05 2007 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=) Date: Tue, 22 May 2007 10:31:05 +0000 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <4652128B.3000301@v.loewis.de> References: <009f01c79b51$b1da52c0$1f0a0a0a@enfoldsystems.local> <46512B09.1080304@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE4B0@exchis.ccp.ad.local> <4652128B.3000301@v.loewis.de> Message-ID: <4E9372E6B2234D4F859320D896059A9508DCBEE6FB@exchis.ccp.ad.local> > -----Original Message----- > Nobody proposed to ditch cross-compilation. I very much like > cross-compilation, I do all Itanium and AMD64 in cross-compilation. > > I just want the *file structure* of the output to be the very same > for all architectures, meaning that they can't coexist in a single > source directory. > Surely there are differences between architectures? PC uses MSI after all. Why can't linux be under trunk/linux and pc 86 under trunk/pcbuild8/win32PGO and 64 under trunk/pcbuild8/x64pgo? We shouldn't let bad tools keep us from new ways of doing things, rather we should fix and update the tools. > No, you use two checkouts, of course. That?s just silly. And two visual studios open, and edit the file in two places too? I say let's just admit that tools can compile for more than one target. Let's adapt to it and we will all be happier. > No matter what the next compiler is (VS 2005 or VS 2007/2008), it's > still *a lot* of work until the VS 2005 build can be used for releasing > Python. For example, there is no support for the SxS installation of > msvcr8.dll, using the MSI merge module. Well, there is some work, which is why I am proposing the Europython sprint to do it. But we are almost there. And that work will be useful for Orcas, when that comes along. And btw, there is no need to install the msvcr8.dll. We can distribute them as a private assembly. then they (and the manifest) exist in the same directory as python2x.dll. This is a supported distribution mode and how we distribute the runtime with EVE. > Not sure whether anything really is needed. Python works fine on Vista. If you are an administrator. A limited user will have problems installing it and then running it. > > > 1) supplying python.dll as a Side By Side assembly > > What would that improve? Well, it should reduce dll-hell problems of applications that ship with python2x.dll. You ship with and link to your own and tested dll. We have some concerns here, for example, now that we are moving away from embedding python in our blue.dll and using python25.dll directly, that this exposes a vulnerability to the integrity of the software. > > > 2) Changing python install locations To conform with Windows rules and get a "Vista approved" logo. Install in the ProgramFiles folder. Just as C does. Ah, and this also means that we could install both 32 bit and 64 bit versions, another plus. > > 3) Supporting shadow libraries, where .pyc files end up in a > different > > hierarchy from the .py files. (useful for many things beside > VISTA) > > Yes, and people had written PEPs for it which failed due to lack of > follow up. Interesting. We are definitely interested in that. You see, Someone installs a game or accounting software using vista. He then runs as a limited user. Python insists on saving its .pyc files in the installation folder, but this is not something that is permitted on Vista. > > > 4) Signing the python dlls and executables > Hmm. Again, easing the installation experience for vista users. So that they don't get a red box about an unknown application requiring administrator privileges. > > Sure, and have they reported problems with Python on Vista (problems > specific to Vista?) Certainly. We are working on them, of course. Chiefly, we have had to change where we save all kinds of temporary files. And we have to be careful that all our .py files ship as .pyc files. And we are versioning, and signing all executables, providing user level manifests and fixing up the install processes to be more compliant. I am just saying that this is something that a standard python distro might want to do too. Kristjan From kristjan at ccpgames.com Tue May 22 13:38:23 2007 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=) Date: Tue, 22 May 2007 11:38:23 +0000 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <017001c79c20$dd347e80$1f0a0a0a@enfoldsystems.local> References: <46520F4F.5010502@v.loewis.de> <017001c79c20$dd347e80$1f0a0a0a@enfoldsystems.local> Message-ID: <4E9372E6B2234D4F859320D896059A9508DCBEE754@exchis.ccp.ad.local> > -----Original Message----- > > > It seems the > > > best thing might be to modify the PCBuild8 build process so the > output > > > binaries are in the ../PCBuild' directory - this way distutils and > others > > > continue to work fine. Does that sound reasonable? > > > I think Kristjan will have to say a word here: I think he just likes > > it the way it is right now. That would rather suggest that build_ext > > needs to be changed. > Someone mentioned the idea to have a bat file do it. I like that idea. There is already a build.bat file which will build whatever configuration you choose (platform and configuration). Extending it to subsequently copy the resulting binaries up one level is easy. Perhaps this is an acceptable compromise? Kristjan From mhammond at skippinet.com.au Tue May 22 15:30:57 2007 From: mhammond at skippinet.com.au (Mark Hammond) Date: Tue, 22 May 2007 23:30:57 +1000 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <4E9372E6B2234D4F859320D896059A9508DCBEE754@exchis.ccp.ad.local> Message-ID: <01d401c79c75$6fd20510$1f0a0a0a@enfoldsystems.local> > Someone mentioned the idea to have a bat file do it. I like > that idea. There is already a build.bat file which will > build whatever configuration you choose (platform and > configuration). Extending it to subsequently copy the > resulting binaries up one level is easy. Perhaps this is an > acceptable compromise? Sure - that will work for me. I'll check this out and contact you by private mail for further guidance. Cheers, Mark -------------- next part -------------- A non-text attachment was scrubbed... Name: winmail.dat Type: application/ms-tnef Size: 1784 bytes Desc: not available Url : http://mail.python.org/pipermail/distutils-sig/attachments/20070522/aba45765/attachment.bin From dobesv at gmail.com Tue May 22 19:19:06 2007 From: dobesv at gmail.com (Dobes Vandermeer) Date: Tue, 22 May 2007 10:19:06 -0700 Subject: [Distutils] mingw and environment variables Message-ID: It seems to be a problem with distutils currently that g++ isn't used to link at the appropriate time. Ideally you'd split linking into C++ and C variations, and detect whether the C++ compiler was used and use the C++ linker if it was. And, a bug related to this, is that you can't easily override the linker when using mingw; the customize_compiler() function checks for a type of 'unix': def customize_compiler(compiler): """Do any platform-specific customization of a CCompiler instance. Mainly needed on Unix, so we can plug in the information that varies across Unices and is stored in Python's Makefile. """ if compiler.compiler_type == "unix": But the mingw compiler (which is unix compatible) has a compiler_type 'mingw32', not 'unix'. The easiest "band-aid" patch would be to use if compiler.compiler_type in ("unix", "mingw32"): but I wonder whether: if isinstance(compiler, UnixCCompiler): would be more appropriate? The workaround I came up with is: # Force g++ for linking import distutils.sysconfig old_customize_compiler = distutils.sysconfig.customize_compiler def customize_compiler(compiler): old_customize_compiler(compiler) if compiler.compiler_type == 'mingw': compiler.set_executables(linker_so='g++ -mno-cygwin -shared') distutils.sysconfig.customize_compiler = customize_compiler From jkp at kirkconsulting.co.uk Tue May 22 21:16:19 2007 From: jkp at kirkconsulting.co.uk (Jamie Kirkpatrick) Date: Tue, 22 May 2007 20:16:19 +0100 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <46520F4F.5010502@v.loewis.de> References: <00af01c79b6f$2f863c30$1f0a0a0a@enfoldsystems.local> <46520F4F.5010502@v.loewis.de> Message-ID: > > I recommend that those people install the official binaries. Why do > you > need to build the binaries from source, if all you want is to build > extensions? I've been following this discussion and it seems like an appropriate place to mention such a scenario which I have encountered myself and am still trying to work around: I have a set of extensions that use SWIG to wrap my own C++ library. This library, on a day-to-day basis is built against VS8 since the rest of our product suite is. Right now I have no way to work with this code using VS8 since the standard distribution is built against VS7 which uses a different CRT. This is an absolute nightmare in practice since I currently have to maintain VS7 projects in parallel with the standard VS8 ones simply so that I can run and test my python code. I've downloaded the Python source and had a look at building up my own distributions for each case (ideally there would be an easy way to separate out Release / Debug products as well as the VS8 / VS7 variants, and I guess potentially for those cross-compiling we'd need to go a step further and do this per arch as well. Anyway, this isn't how it works at the moment, but I'm still searching for a way to be able to work on the python code in VS8. Building using the current projects I seem to get everything in the PCBuild8 / PCBuild dirs. How can I work with what is build? Is there a shell script to build a final distribution tree? If not, is there a simple way to build an MSI similar to the one found on the Python.org site for the official releases but using the PCBuild8 stuff? If not how do you recommend getting myself to a state where I have at least a feature complete distribution build against VS8? I'm happy with a one time build that I can just install into my source tree and upload to the SCM. Thanks in advance, and I hope that my thoughts proved useful in some way. Jamie -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20070522/d5f8d368/attachment.html From sklein at cpcug.org Tue May 22 23:01:28 2007 From: sklein at cpcug.org (Stanley A. Klein) Date: Tue, 22 May 2007 17:01:28 -0400 (EDT) Subject: [Distutils] I need to create a customized Fedora live cd with Python applications on it Message-ID: <24685.207.188.248.157.1179867688.squirrel@www.cpcug.org> On 4/24/07, "Nathan R. Yergler" wrote: >On 4/24/07, Stanley A. Klein wrote: >> I need to create a customized Fedora live CD with Python applications on >> it. The live CD requires rpm's of the applications. (The issues are >> similar to a customized Ubuntu live CD but Ubuntu requires deb's.) >> >> It seems that the distutils/setup.py functions that create rpms are >> broken. I get errors most of the time that I try to use them. The >Python >> community seems to be focused on eggs, that are useful if individually >> installed but don't match the distribution formats of the Linux distros. >> Those distribution formats are what the distros use for their live CD's. >I've successfully created RPMs many times in the past; can you >elaborate on the problems you're having? >> >> Is there a way to convert eggs to rpm's? Can the distutils be fixed to >> reliably create rpm's? Is there some other way to install eggs on distro >> customized live CD's? >If it's a live CD, is there a reason you can't just put them into >site-packages on the CD filesystem? (I have no idea what goes into >building a Fedora live CD, so "yes, there's a reason" is a sufficient >answer). Below is a typical example of the problem I encounter. I tried to build an RPM of ZSI-2.0 using "python setup.py bdist_rpm", and got the output below at the end. Most of the time I have the same problem trying to create RPM's from python tar.gz files. The reason you can't just put the files into site-packages on the CD file system (AFAIK for either Fedora or a Debian derivative such as Ubuntu) is that the live CD builds its system from packages in the distro format. To get files into site-packages as part of the live CD, they have to be loaded there from an RPM or deb format package. I don't know of any way the live CD's can be set up to handle eggs. Also, the live CD's are now being made installable, so if the user wants to do so, installing from the live CD is an option. The installers expect distro formatted packages. Stan Klein ---------------------------------------------------------------------- error: Installed (but unpackaged) file(s) found: /usr/lib/python2.4/site-packages/ZSI/ServiceContainer.pyo /usr/lib/python2.4/site-packages/ZSI/ServiceProxy.pyo /usr/lib/python2.4/site-packages/ZSI/TC.pyo /usr/lib/python2.4/site-packages/ZSI/TCapache.pyo /usr/lib/python2.4/site-packages/ZSI/TCcompound.pyo /usr/lib/python2.4/site-packages/ZSI/TCnumbers.pyo /usr/lib/python2.4/site-packages/ZSI/TCtimes.pyo /usr/lib/python2.4/site-packages/ZSI/__init__.pyo /usr/lib/python2.4/site-packages/ZSI/address.pyo /usr/lib/python2.4/site-packages/ZSI/auth.pyo /usr/lib/python2.4/site-packages/ZSI/client.pyo /usr/lib/python2.4/site-packages/ZSI/digest_auth.pyo /usr/lib/python2.4/site-packages/ZSI/dispatch.pyo /usr/lib/python2.4/site-packages/ZSI/fault.pyo /usr/lib/python2.4/site-packages/ZSI/generate/__init__.pyo /usr/lib/python2.4/site-packages/ZSI/generate/commands.pyo /usr/lib/python2.4/site-packages/ZSI/generate/containers.pyo /usr/lib/python2.4/site-packages/ZSI/generate/pyclass.pyo /usr/lib/python2.4/site-packages/ZSI/generate/utility.pyo /usr/lib/python2.4/site-packages/ZSI/generate/wsdl2dispatch.pyo /usr/lib/python2.4/site-packages/ZSI/generate/wsdl2python.pyo /usr/lib/python2.4/site-packages/ZSI/parse.pyo /usr/lib/python2.4/site-packages/ZSI/resolvers.pyo /usr/lib/python2.4/site-packages/ZSI/schema.pyo /usr/lib/python2.4/site-packages/ZSI/twisted/WSresource.pyo /usr/lib/python2.4/site-packages/ZSI/twisted/WSsecurity.pyo /usr/lib/python2.4/site-packages/ZSI/twisted/__init__.pyo /usr/lib/python2.4/site-packages/ZSI/twisted/client.pyo /usr/lib/python2.4/site-packages/ZSI/typeinterpreter.pyo /usr/lib/python2.4/site-packages/ZSI/version.pyo /usr/lib/python2.4/site-packages/ZSI/writer.pyo /usr/lib/python2.4/site-packages/ZSI/wstools/Namespaces.pyo /usr/lib/python2.4/site-packages/ZSI/wstools/TimeoutSocket.pyo /usr/lib/python2.4/site-packages/ZSI/wstools/UserTuple.pyo /usr/lib/python2.4/site-packages/ZSI/wstools/Utility.pyo /usr/lib/python2.4/site-packages/ZSI/wstools/WSDLTools.pyo /usr/lib/python2.4/site-packages/ZSI/wstools/XMLSchema.pyo /usr/lib/python2.4/site-packages/ZSI/wstools/XMLname.pyo /usr/lib/python2.4/site-packages/ZSI/wstools/__init__.pyo /usr/lib/python2.4/site-packages/ZSI/wstools/c14n.pyo /usr/lib/python2.4/site-packages/ZSI/wstools/logging.pyo RPM build errors: File listed twice: /usr/lib/python2.4/site-packages/ZSI-2.0-py2.4.egg-info/PKG-INFO File listed twice: /usr/lib/python2.4/site-packages/ZSI-2.0-py2.4.egg-info/SOURCES.txt File listed twice: /usr/lib/python2.4/site-packages/ZSI-2.0-py2.4.egg-info/dependency_links.txt File listed twice: /usr/lib/python2.4/site-packages/ZSI-2.0-py2.4.egg-info/entry_points.txt File listed twice: /usr/lib/python2.4/site-packages/ZSI-2.0-py2.4.egg-info/requires.txt File listed twice: /usr/lib/python2.4/site-packages/ZSI-2.0-py2.4.egg-info/top_level.txt Installed (but unpackaged) file(s) found: /usr/lib/python2.4/site-packages/ZSI/ServiceContainer.pyo /usr/lib/python2.4/site-packages/ZSI/ServiceProxy.pyo /usr/lib/python2.4/site-packages/ZSI/TC.pyo /usr/lib/python2.4/site-packages/ZSI/TCapache.pyo /usr/lib/python2.4/site-packages/ZSI/TCcompound.pyo /usr/lib/python2.4/site-packages/ZSI/TCnumbers.pyo /usr/lib/python2.4/site-packages/ZSI/TCtimes.pyo /usr/lib/python2.4/site-packages/ZSI/__init__.pyo /usr/lib/python2.4/site-packages/ZSI/address.pyo /usr/lib/python2.4/site-packages/ZSI/auth.pyo /usr/lib/python2.4/site-packages/ZSI/client.pyo /usr/lib/python2.4/site-packages/ZSI/digest_auth.pyo /usr/lib/python2.4/site-packages/ZSI/dispatch.pyo /usr/lib/python2.4/site-packages/ZSI/fault.pyo /usr/lib/python2.4/site-packages/ZSI/generate/__init__.pyo /usr/lib/python2.4/site-packages/ZSI/generate/commands.pyo /usr/lib/python2.4/site-packages/ZSI/generate/containers.pyo /usr/lib/python2.4/site-packages/ZSI/generate/pyclass.pyo /usr/lib/python2.4/site-packages/ZSI/generate/utility.pyo /usr/lib/python2.4/site-packages/ZSI/generate/wsdl2dispatch.pyo /usr/lib/python2.4/site-packages/ZSI/generate/wsdl2python.pyo /usr/lib/python2.4/site-packages/ZSI/parse.pyo /usr/lib/python2.4/site-packages/ZSI/resolvers.pyo /usr/lib/python2.4/site-packages/ZSI/schema.pyo /usr/lib/python2.4/site-packages/ZSI/twisted/WSresource.pyo /usr/lib/python2.4/site-packages/ZSI/twisted/WSsecurity.pyo /usr/lib/python2.4/site-packages/ZSI/twisted/__init__.pyo /usr/lib/python2.4/site-packages/ZSI/twisted/client.pyo /usr/lib/python2.4/site-packages/ZSI/typeinterpreter.pyo /usr/lib/python2.4/site-packages/ZSI/version.pyo /usr/lib/python2.4/site-packages/ZSI/writer.pyo /usr/lib/python2.4/site-packages/ZSI/wstools/Namespaces.pyo /usr/lib/python2.4/site-packages/ZSI/wstools/TimeoutSocket.pyo /usr/lib/python2.4/site-packages/ZSI/wstools/UserTuple.pyo /usr/lib/python2.4/site-packages/ZSI/wstools/Utility.pyo /usr/lib/python2.4/site-packages/ZSI/wstools/WSDLTools.pyo /usr/lib/python2.4/site-packages/ZSI/wstools/XMLSchema.pyo /usr/lib/python2.4/site-packages/ZSI/wstools/XMLname.pyo /usr/lib/python2.4/site-packages/ZSI/wstools/__init__.pyo /usr/lib/python2.4/site-packages/ZSI/wstools/c14n.pyo /usr/lib/python2.4/site-packages/ZSI/wstools/logging.pyo error: command 'rpmbuild' failed with exit status 1 From pje at telecommunity.com Tue May 22 23:03:42 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 22 May 2007 17:03:42 -0400 Subject: [Distutils] I need to create a customized Fedora live cd with Python applications on it In-Reply-To: <24685.207.188.248.157.1179867688.squirrel@www.cpcug.org> References: <24685.207.188.248.157.1179867688.squirrel@www.cpcug.org> Message-ID: <20070522210151.AB6BE3A4061@sparrow.telecommunity.com> At 05:01 PM 5/22/2007 -0400, Stanley A. Klein wrote: >Below is a typical example of the problem I encounter. I tried to build >an RPM of ZSI-2.0 using "python setup.py bdist_rpm", and got the output >below at the end. What version of setuptools are you using? From sklein at cpcug.org Tue May 22 23:23:51 2007 From: sklein at cpcug.org (Stanley A. Klein) Date: Tue, 22 May 2007 17:23:51 -0400 (EDT) Subject: [Distutils] I need to create a customized Fedora live cd with Python applications on it In-Reply-To: <20070522210151.AB6BE3A4061@sparrow.telecommunity.com> References: <24685.207.188.248.157.1179867688.squirrel@www.cpcug.org> <20070522210151.AB6BE3A4061@sparrow.telecommunity.com> Message-ID: <38000.207.188.248.157.1179869031.squirrel@www.cpcug.org> Prior to attempting the rpm build, I updated my version of setuptools to 0.6c5 (package python-setuptools-0.6c5-1.fc5). Stan Klein On Tue, May 22, 2007 5:03 pm, Phillip J. Eby wrote: > At 05:01 PM 5/22/2007 -0400, Stanley A. Klein wrote: >>Below is a typical example of the problem I encounter. I tried to build >>an RPM of ZSI-2.0 using "python setup.py bdist_rpm", and got the output >>below at the end. > > What version of setuptools are you using? > From martin at v.loewis.de Tue May 22 23:30:21 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 22 May 2007 23:30:21 +0200 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <20070522055257.GA13609@caltech.edu> References: <009f01c79b51$b1da52c0$1f0a0a0a@enfoldsystems.local> <46512B09.1080304@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE4B0@exchis.ccp.ad.local> <46527929.8040600@holdenweb.com> <4652807A.2030707@v.loewis.de> <20070522055257.GA13609@caltech.edu> Message-ID: <465360ED.2090000@v.loewis.de> > I'd be happy to install Windows and the latest VisualStudio on a 64-bit > VMware image. I just can't be responsible for day-to-day administration > of the buildslave; Windows requires too much attention for me to do > that. Thanks for the offer. Perhaps Kristjan is interested in setting up a buildslave on such an installation. Regards, Martin From martin at v.loewis.de Tue May 22 23:37:20 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 22 May 2007 23:37:20 +0200 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <019101c79c3a$91d06b60$1f0a0a0a@enfoldsystems.local> References: <019101c79c3a$91d06b60$1f0a0a0a@enfoldsystems.local> Message-ID: <46536290.8050102@v.loewis.de> > While that is true in theory, I often find it is not the case in practice, > generally due to the optimizer. Depending on what magic the compiler has > done, it can be very difficult to set breakpoints (conditional or > otherwise), inspect variable values, etc. It is useful in some cases, but > very often I find myself falling back to a debug build after attempting to > debug a local release build. What I typically do is to disable optimization in a release build. It then essentially becomes a debug build, except that _DEBUG is not defined, so that it doesn't incorporate the debug CRT. This, in turn, means that you can mix such a "debug" build readily with non-debug binaries. >> I think there are several other cases in distutils to support >> special cases (e.g. the DISTUTILS_USE_SDK environment variable); >> if people want to see their setup supported, AND ARE WILLING TO >> CONTRIBUTE: more power to them. > > Yes, but I believe its also important to solicit feedback on the best way to > achieve their aims. In this particular case, I believe it would have been > misguided for me to simply check in whatever was necessary to have distutils > work from the PCBuild8 directory. I hope it is clear that I am willing to > contribute the outcome of these discussions... Sure - all understood, and all fine. It's certainly good to build consensus first, but in cases of "should we support that borderline case of build setup as long as I'm willing to maintain it", the answer should always be "yes" (*), irrespective of the merits of the specific proposal. Heck, we would even accept patches to support HP-UX better :-) Regards, Martin (*) assuming it doesn't break anything P.S. I think there have been cases where I haven't been that supportive wrt. build issues. However, in this case I really feel "go ahead". From martin at v.loewis.de Tue May 22 23:51:26 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 22 May 2007 23:51:26 +0200 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <4E9372E6B2234D4F859320D896059A9508DCBEE6FB@exchis.ccp.ad.local> References: <009f01c79b51$b1da52c0$1f0a0a0a@enfoldsystems.local> <46512B09.1080304@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE4B0@exchis.ccp.ad.local> <4652128B.3000301@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE6FB@exchis.ccp.ad.local> Message-ID: <465365DE.1090306@v.loewis.de> > Surely there are differences between architectures? PC uses MSI after all. > Why can't linux be under trunk/linux and pc 86 under trunk/pcbuild8/win32PGO > and 64 under trunk/pcbuild8/x64pgo? That couldn't work for me. I try avoid building on a network drive, and with local drives, I just can't have a Windows build and a Linux build on the same checkout - they live on separate file systems, after all (Linux on ext3, Windows on NTFS, with multi-boot switching between them). > That?s just silly. And two visual studios open, and edit the file > in two places too? I have about 10 checkouts of Python, on different machines, with no problems. I don't feel silly doing so. I don't *use* them simultaneously, of course - I cannot work on two architectures simultaneously, anyway. > I say let's just admit that tools can compile for > more than one target. Let's adapt to it and we will all be happier. You might be; I will be sad. It comes for a price, and with little benefit. Disk space is cheaper than my time to fight build processes. > And btw, there is no need to install the msvcr8.dll. We can distribute > them as a private assembly. then they (and the manifest) exist in the same > directory as python2x.dll. Yes, but then python2x.dll goes into system32, and so will msvcr8.dll, no? >> Not sure whether anything really is needed. Python works fine on Vista. > If you are an administrator. A limited user will have problems installing > it and then running it. Is there a bug report for that? >>> 1) supplying python.dll as a Side By Side assembly >> What would that improve? > Well, it should reduce dll-hell problems of applications that ship with > python2x.dll. You ship with and link to your own and tested dll. We > have some concerns here, for example, now that we are moving away from > embedding python in our blue.dll and using python25.dll directly, that > this exposes a vulnerability to the integrity of the software. Why should there be versioning problems with python25.dll? Are there any past issues with incompatibilities with any python2x.dll release? >>> 2) Changing python install locations > To conform with Windows rules and get a "Vista approved" logo. > Install in the ProgramFiles folder. Only over my dead body. *This* is silly. > Just as C does. Ah, and > this also means that we could install both 32 bit and 64 bit > versions, another plus. What about the registry? > Interesting. We are definitely interested in that. You see, Someone > installs a game or accounting software using vista. He then runs as a > limited user. Python insists on saving its .pyc files in the installation > folder, but this is not something that is permitted on Vista. But that's not a problem, is it? Writing silently "fails", i.e. it just won't save the pyc files. Happens all the time on Unix. >> Sure, and have they reported problems with Python on Vista (problems >> specific to Vista?) > Certainly. We are working on them, of course. But, of course, they have not been reported. Regards, Martin From martin at v.loewis.de Tue May 22 23:58:31 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 22 May 2007 23:58:31 +0200 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: References: <00af01c79b6f$2f863c30$1f0a0a0a@enfoldsystems.local> <46520F4F.5010502@v.loewis.de> Message-ID: <46536787.7040500@v.loewis.de> > I have a set of extensions that use SWIG to wrap my own C++ library. > This library, on a day-to-day basis is built against VS8 since the rest > of our product suite is. Right now I have no way to work with this code > using VS8 since the standard distribution is built against VS7 which > uses a different CRT. This is an absolute nightmare in practice since > I currently have to maintain VS7 projects in parallel with the standard > VS8 ones simply so that I can run and test my python code. If you know well enough what you are doing, and avoid using unsupported cases, you *can* mix different CRTs. > Building using the current projects I > seem to get everything in the PCBuild8 / PCBuild dirs. How can I work > with what is build? Python works correctly out of its build directory. You can just run it from there. > Is there a shell script to build a final distribution tree? Not sure what you mean by "tree". Distribution is in a single MSI file, not in a tree, and the tree that gets created on the target machine (of the MSI file) never exists on the source machine. > If not, is > there a simple way to build an MSI similar to the one found on the > Python.org site for the official releases but using the PCBuild8 stuff? No, but it should be possible to adopt Tools/msi to package pcbuild8 instead. Just make sure you are using different GUIDs as the ProductCode. Regards, Martin From pje at telecommunity.com Wed May 23 00:02:36 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 22 May 2007 18:02:36 -0400 Subject: [Distutils] I need to create a customized Fedora live cd with Python applications on it In-Reply-To: <38000.207.188.248.157.1179869031.squirrel@www.cpcug.org> References: <24685.207.188.248.157.1179867688.squirrel@www.cpcug.org> <20070522210151.AB6BE3A4061@sparrow.telecommunity.com> <38000.207.188.248.157.1179869031.squirrel@www.cpcug.org> Message-ID: <20070522220044.C82EA3A4061@sparrow.telecommunity.com> At 05:23 PM 5/22/2007 -0400, Stanley A. Klein wrote: >Prior to attempting the rpm build, I updated my version of setuptools to >0.6c5 (package python-setuptools-0.6c5-1.fc5). Would you mind emailing me the *entire* output of the bdist_rpm operation? When I try this with 0.6c5 and the source of ZSI-2.0-rc3, I don't get the errors (although I do see the "files listed twice" warnings). Thanks. From sklein at cpcug.org Wed May 23 04:23:59 2007 From: sklein at cpcug.org (Stanley A. Klein) Date: Tue, 22 May 2007 22:23:59 -0400 (EDT) Subject: [Distutils] I need to create a customized Fedora live cd with Python applications on it In-Reply-To: <20070522220044.C82EA3A4061@sparrow.telecommunity.com> References: <24685.207.188.248.157.1179867688.squirrel@www.cpcug.org> <20070522210151.AB6BE3A4061@sparrow.telecommunity.com> <38000.207.188.248.157.1179869031.squirrel@www.cpcug.org> <20070522220044.C82EA3A4061@sparrow.telecommunity.com> Message-ID: <55520.71.163.214.198.1179887039.squirrel@www.cpcug.org> I'll have to try it with rc3 and see if I get the same errors (although I get that behavior very frequently when trying to build rpm's). I was using (final release) ZSI-2.0 downloaded from pywebsvcs.sourceforge.net. I'll do it tomorrow in the office and send you both. Stan Klein On Tue, May 22, 2007 6:02 pm, Phillip J. Eby wrote: > At 05:23 PM 5/22/2007 -0400, Stanley A. Klein wrote: >>Prior to attempting the rpm build, I updated my version of setuptools to >>0.6c5 (package python-setuptools-0.6c5-1.fc5). > > Would you mind emailing me the *entire* output of the bdist_rpm > operation? When I try this with 0.6c5 and the source of ZSI-2.0-rc3, > I don't get the errors (although I do see the "files listed twice" > warnings). Thanks. > -- From martin at v.loewis.de Tue May 22 23:39:18 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 22 May 2007 23:39:18 +0200 Subject: [Distutils] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <4E9372E6B2234D4F859320D896059A9508DCBEE6D6@exchis.ccp.ad.local> References: <009f01c79b51$b1da52c0$1f0a0a0a@enfoldsystems.local> <46512B09.1080304@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE4B0@exchis.ccp.ad.local> <46527929.8040600@holdenweb.com> <4652807A.2030707@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE6D6@exchis.ccp.ad.local> Message-ID: <46536306.7080108@v.loewis.de> > If there are no technical and corporate difficulties, such as > firewalls and security, I am sure that CCP can provide a > VisualStudio2005 buildbot for our needs. Wasn't there some issue that > each buildbot can only provide a single build? Yes, but you can have multiple buildbot slaves on a single machine. They all create separate checkouts. Regards, Martin From kristjan at ccpgames.com Wed May 23 18:38:06 2007 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=) Date: Wed, 23 May 2007 16:38:06 +0000 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <465365DE.1090306@v.loewis.de> References: <009f01c79b51$b1da52c0$1f0a0a0a@enfoldsystems.local> <46512B09.1080304@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE4B0@exchis.ccp.ad.local> <4652128B.3000301@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE6FB@exchis.ccp.ad.local> <465365DE.1090306@v.loewis.de> Message-ID: <4E9372E6B2234D4F859320D896059A9508DCBEEACB@exchis.ccp.ad.local> > -----Original Message----- > From: "Martin v. L?wis" [mailto:martin at v.loewis.de] > > That couldn't work for me. I try avoid building on a network drive, and > with local drives, I just can't have a Windows build and a Linux build > on the same checkout - they live on separate file systems, after all > (Linux on ext3, Windows on NTFS, with multi-boot switching between > them). Very well, leaving linux aside, I don't see why this: /win32mount/trunk/PCbuild/ /x64mount/trunk/PCbuild/ Is any different from /winmount/trunk/PCBuild/win32 /winmount/trunk/PCBuild/x64 I don't understand this extraordinary reluctance to add a single extra directory. The windows build process is different from any other build process, so even If all the other platforms live one directory higher, why must windows? > I don't *use* them simultaneously, of course - I cannot work > on two architectures simultaneously, anyway. I do so on a daily basis. I designed PCBuild8 to be easy for interactive work using VisualStudio, using property sheets and such to group common settings for easy editing. During the course of my work, I will edit a file (which is checked out from Perforce), compile debug for two platforms, test, and repeat. > > > I say let's just admit that tools can compile for > > more than one target. Let's adapt to it and we will all be happier. > > You might be; I will be sad. It comes for a price, I well understand the benefits, I use it all the time, but the price still eludes me. how can a different name for the output folder for a different platform be such a big problem? > > And btw, there is no need to install the msvcr8.dll. We can > distribute > > them as a private assembly. then they (and the manifest) exist in > the same > > directory as python2x.dll. > > Yes, but then python2x.dll goes into system32, and so will msvcr8.dll, > no? Yes, that is correct. Well, there is a CRT .exe redist if you want to deploy this into the SxS cache, it just has to be run as part of the install process. But that may or may not be problematic, I don't know. > > >> Not sure whether anything really is needed. Python works fine on > Vista. > > If you are an administrator. A limited user will have problems > installing > > it and then running it. > > Is there a bug report for that? I don't know. At any rate, I think any vista issues is a completely separate thing, something that needs to be handled as a whole, rather than responding to a particular problem reported in a bug report. > > >>> 1) supplying python.dll as a Side By Side assembly > >> What would that improve? > > Well, it should reduce dll-hell problems of applications that ship > with > > python2x.dll. You ship with and link to your own and tested dll. We > > have some concerns here, for example, now that we are moving away > from > > embedding python in our blue.dll and using python25.dll directly, > that > > this exposes a vulnerability to the integrity of the software. > > Why should there be versioning problems with python25.dll? Are there > any past issues with incompatibilities with any python2x.dll release? Someone could replace the python25.dll that we ship with their own patched version, thereby gaining backdoor access to the software. The way windows searches for old style dlls makes this easy. Using the SxS signed loading scheme, you can protect yourself up to a point from such attacks. Of course, this doesn't have to be a problem for vanilla python, we can do this on our own for the patched python25 that we employ, but it still might be something others could find useful. > > Install in the ProgramFiles folder. > > Only over my dead body. *This* is silly. Bill doesn't think so. And he gets to decide. I mean we do want to play nice, don't we? Nothing installs itself in the root anymore, not since windows 3.1 > > > Just as C does. Ah, and > > this also means that we could install both 32 bit and 64 bit > > versions, another plus. > > What about the registry? I don't know about the registry, what is it used for? 64 bit windows already ships with dual versions of some apps, notably explorer.exe so that shouldn't be a big problem. > > > Interesting. We are definitely interested in that. You see, Someone > > installs a game or accounting software using vista. He then runs as > a > > limited user. Python insists on saving its .pyc files in the > installation > > folder, but this is not something that is permitted on Vista. > > But that's not a problem, is it? Writing silently "fails", i.e. it just > won't save the pyc files. Happens all the time on Unix. It may not silently fail, depending on your user status. An admin might get a confirmation window, for example. > > >> Sure, and have they reported problems with Python on Vista (problems > >> specific to Vista?) > > Certainly. We are working on them, of course. > > But, of course, they have not been reported. These are not python errors as such, but rather EVE errors. We ship the .py files precompiled and therefore avoid the aforementioned problems. But we have had to move all sorts of temporary files out of "program files" and into "documents and settings/user/local settings/Application Data/CCP/Eve Kristjan From trentm at activestate.com Wed May 23 21:27:48 2007 From: trentm at activestate.com (Trent Mick) Date: Wed, 23 May 2007 12:27:48 -0700 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <46520F4F.5010502@v.loewis.de> References: <00af01c79b6f$2f863c30$1f0a0a0a@enfoldsystems.local> <46520F4F.5010502@v.loewis.de> Message-ID: <465495B4.1040400@activestate.com> [MarkH] >> I'm guessing vsextcomp doesn't use the Visual >> Studio 'ReleaseAMD64' configuration - would it be OK for me to check in >> changes to the PCBuild projects for this configuration? > [Martin v. L?wis] > Please don't. It exclusively relies on vsextcomp, and is only useful > if you have that infrastructure installed. See for yourself: it uses > the /USE_CL:MS_OPTERON command line switch, which isn't a Microsoft > invention (but instead invented by Peter Tr?ger). Aside: it isn't my experience that vsextcomp is necessary to cross-compile for AMD64 and IA64. My cross-build process basically equates to: - Run the appropriate environment setup for the correct compiler. E.g., for the Platform SDK AMD64 compiler and with the current Platform SDK this is: C:\Program Files\Microsoft Platform SDK\SetEnv.Cmd /X64 /RETAIL - Run the solution file with "devenv.com" (IIRC, devenv.exe doesn't take command-line args) and be sure to pass ing "/useenv" to pick up the environment changes. (*) set DEVENV_COM=path/to/devenv.com %DEVENV_COM% PCbuild\pcbuild.sln /useenv /build ReleaseAMD64 (*) Note that for a cross-build the "make_versioninfo" and "make_buildinfo" projects need to be built natively first: "C:\Program Files\Microsoft Visual Studio .NET 2003\Vc7\bin\vcvars32.bat" %DEVENV_COM% PCbuild\pcbuild.sln /useenv /build ReleaseAMD64 /project make_versioninfo %DEVENV_COM% PCbuild\pcbuild.sln /useenv /build ReleaseAMD64 /project make_buildinfo This is all VS7.1, though. I don't yet know if VS8 throws a spanner into the works. For VS6 I use "msdev" instead of "devenv.com" and "PC\VC6\pcbuild.dsw" instead of "PCbuild\pcbuild.sln". I haven't looked into what vsextcomp does, so apologies if this is ignorant. Trent -- Trent Mick trentm at activestate.com From trentm at activestate.com Wed May 23 21:37:54 2007 From: trentm at activestate.com (Trent Mick) Date: Wed, 23 May 2007 12:37:54 -0700 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <46520F4F.5010502@v.loewis.de> References: <00af01c79b6f$2f863c30$1f0a0a0a@enfoldsystems.local> <46520F4F.5010502@v.loewis.de> Message-ID: <46549812.9080400@activestate.com> >> It seems the >> best thing might be to modify the PCBuild8 build process so the output >> binaries are in the ../PCBuild' directory - this way distutils and others >> continue to work fine. Does that sound reasonable? > > I think Kristjan will have to say a word here: I think he just likes > it the way it is right now. That would rather suggest that build_ext > needs to be changed. I use this patch in ActivePython to get distutils to find the correct PCbuild dir (see attached). Trent -- Trent Mick trentm at activestate.com -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: distutils_pcbuild_lib_dir.patch Url: http://mail.python.org/pipermail/distutils-sig/attachments/20070523/41c69cef/attachment.asc From snaury at gmail.com Wed May 23 22:36:21 2007 From: snaury at gmail.com (Alexey Borzenkov) Date: Thu, 24 May 2007 00:36:21 +0400 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <4E9372E6B2234D4F859320D896059A9508DCBEEACB@exchis.ccp.ad.local> References: <009f01c79b51$b1da52c0$1f0a0a0a@enfoldsystems.local> <46512B09.1080304@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE4B0@exchis.ccp.ad.local> <4652128B.3000301@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE6FB@exchis.ccp.ad.local> <465365DE.1090306@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEEACB@exchis.ccp.ad.local> Message-ID: On 5/23/07, Kristj?n Valur J?nsson wrote: > > > Install in the ProgramFiles folder. > > Only over my dead body. *This* is silly. > Bill doesn't think so. And he gets to decide. I mean we do want > to play nice, don't we? Nothing installs itself in the root anymore, > not since windows 3.1 Maybe installing in the root is not good, but installing to "Program Files" is just asking for trouble. All sorts of development tools might suddenly break because of that space in the middle of the path and requirement to use quotes around it. I thus usually install things to :\Programs. I'm not sure if any packages/programs will break because of that space, but what if some will? From martin at v.loewis.de Thu May 24 00:34:07 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 24 May 2007 00:34:07 +0200 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <4E9372E6B2234D4F859320D896059A9508DCBEEACB@exchis.ccp.ad.local> References: <009f01c79b51$b1da52c0$1f0a0a0a@enfoldsystems.local> <46512B09.1080304@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE4B0@exchis.ccp.ad.local> <4652128B.3000301@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE6FB@exchis.ccp.ad.local> <465365DE.1090306@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEEACB@exchis.ccp.ad.local> Message-ID: <4654C15F.2040906@v.loewis.de> > Very well, leaving linux aside, I don't see why this: > /win32mount/trunk/PCbuild/ > /x64mount/trunk/PCbuild/ > > Is any different from > /winmount/trunk/PCBuild/win32 > /winmount/trunk/PCBuild/x64 > > I don't understand this extraordinary reluctance to add a single extra directory. > The windows build process is different from any other build process, so even > If all the other platforms live one directory higher, why must windows? It doesn't need to, and reluctance is not wrt. to the proposed new layout, but wrt. changing the current one. Tons of infrastructure depends on the files having exactly the names that they have now, and being located in exactly the locations where they are currently located. Any change to that, whatever minor, will cause problems to some people. So for the change to be made, the advantages must clearly outweigh the incompatibility of the change in the first place. >>> I say let's just admit that tools can compile for >>> more than one target. Let's adapt to it and we will all be happier. >> You might be; I will be sad. It comes for a price, > I well understand the benefits, I use it all the time, but the price > still eludes me. how can a different name for the output folder > for a different platform be such a big problem? When running Python now, I type (after having changed to the source directory in the cmd.exe window) PCbpyt or some such. To navigate to a subdirectory, I need many more keystrokes. I find it very painful to invoke python.exe from the PCbuild8 directory. I don't want that pain to be the default. > Yes, that is correct. Well, there is a CRT .exe redist if you want to deploy > this into the SxS cache, it just has to be run as part of the install process. > But that may or may not be problematic, I don't know. Microsoft recommends to use the merge module (.msm), and I think this is what should be done (if feasible). >> Why should there be versioning problems with python25.dll? Are there >> any past issues with incompatibilities with any python2x.dll release? > Someone could replace the python25.dll that we ship with their own patched > version, thereby gaining backdoor access to the software. The way > windows searches for old style dlls makes this easy. Using the SxS > signed loading scheme, you can protect yourself up to a point from such > attacks. Of course, this doesn't have to be a problem for vanilla > python, we can do this on our own for the patched python25 that we > employ, but it still might be something others could find useful. I personally think that if hostile users can replace DLLs on your system, you have bigger problems than SxS installation of pythhonxy.dll, but perhaps that's just me. >>> Install in the ProgramFiles folder. >> Only over my dead body. *This* is silly. > Bill doesn't think so. And he gets to decide. He can decide not to give Python the "Vista ready" logo. If so, I don't want it. He cannot decide to make the Python installer to install into a directory with a space in its name by default. > I mean we do want to play nice, don't we? Nice to users of Python, sure. I would not have said a word if the standard directory would have been, say "\usr\bin". However, they happened to chose "Program Files", making it language dependent, and putting a space in the name. That space has caused numerous problems to Python scripts, and I expect changing the default would cause a lot of problems to end users. >> What about the registry? > I don't know about the registry, what is it used for? For two things, with different importance to different users: 1. File extensions are registered there, e.g. .py and .pyc. With two binaries installed, the will stomp over each other's file associations; only one of them can win. 2. Python installs keys under HL{LM|CU}\Software\Python\PythonCore\, namely InstallPath InstallGroup PythonPath Documentation Modules For some of these, add-on libraries and applications may modify these keys, and the interpreter will pick up the changes. Again, there can be only one installation on the system "owning" these settings; two simultaneous installations will stomp on each other's settings. > 64 bit windows already ships with dual versions of some apps, notably > explorer.exe so that shouldn't be a big problem. The two versions of MSIE actually *are* a big problem, that's why MS only runs the 32-bit IE, even on Win64 (otherwise, ActiveX controls downloaded from the net wouldn't work). Also, while they are both shipped, you can't the two versions of explorer.exe simultaneously (without trickery), so its far from simple. >> But that's not a problem, is it? Writing silently "fails", i.e. it just >> won't save the pyc files. Happens all the time on Unix. > It may not silently fail, depending on your user status. An admin might > get a confirmation window, for example. Can you describe the precise scenario which makes that happen? To my knowledge, Vista will *not* open a confirmation window when Python attempts to write a .pyc file. Regards, Martin From martin at v.loewis.de Thu May 24 00:45:33 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 24 May 2007 00:45:33 +0200 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <46549812.9080400@activestate.com> References: <00af01c79b6f$2f863c30$1f0a0a0a@enfoldsystems.local> <46520F4F.5010502@v.loewis.de> <46549812.9080400@activestate.com> Message-ID: <4654C40D.1090803@v.loewis.de> > I use this patch in ActivePython to get distutils to find the correct > PCbuild dir (see attached). Would you like to commit this to 2.6? (or perhaps 2.5 even?) Regards, Martin From martin at v.loewis.de Thu May 24 00:48:49 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 24 May 2007 00:48:49 +0200 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <465495B4.1040400@activestate.com> References: <00af01c79b6f$2f863c30$1f0a0a0a@enfoldsystems.local> <46520F4F.5010502@v.loewis.de> <465495B4.1040400@activestate.com> Message-ID: <4654C4D1.3010303@v.loewis.de> > - Run the appropriate environment setup for the correct compiler. E.g., > for the Platform SDK AMD64 compiler and with the current Platform SDK > this is: > > C:\Program Files\Microsoft Platform SDK\SetEnv.Cmd /X64 /RETAIL > > - Run the solution file with "devenv.com" (IIRC, devenv.exe doesn't take > command-line args) and be sure to pass ing "/useenv" to pick up the > environment changes. (*) > > set DEVENV_COM=path/to/devenv.com > %DEVENV_COM% PCbuild\pcbuild.sln /useenv /build ReleaseAMD64 Yes, that should work equally fine. > I haven't looked into what vsextcomp does, so apologies if this is ignorant. It spares you having to setup the environment; it provides a cl.exe wrapper that locates the SDK from the registry, and then invokes the cl.exe in the SDK if necessary. As a consequence, you can still just double-click the solution file, without having to run devenv.exe/com manually. Regards, Martin From mhammond at skippinet.com.au Thu May 24 00:53:06 2007 From: mhammond at skippinet.com.au (Mark Hammond) Date: Thu, 24 May 2007 08:53:06 +1000 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <4E9372E6B2234D4F859320D896059A9508DCBEEACB@exchis.ccp.ad.local> Message-ID: <031801c79d8d$2107b010$1f0a0a0a@enfoldsystems.local> > Very well, leaving linux aside, I don't see why this: > /win32mount/trunk/PCbuild/ > /x64mount/trunk/PCbuild/ > > Is any different from > /winmount/trunk/PCBuild/win32 > /winmount/trunk/PCBuild/x64 In the former case, assuming python is running from the 'trunk' directory, all architectures know how to locate the binary directory - it is always named 'PCBuild'. In the latter case, the directory name depends on the architecture. A number of existing tools already know about the 'PCBuild' directory, so these tools will not need to be taught anything new for x64. > I don't understand this extraordinary reluctance to add a > single extra directory. I think that this thread has enumerated the concerns fairly well. You may not agree with them, but if you don't understand them it might be worth re-reading Martin's responses. Note that I also understand your concerns and goals - I certainly see where you are coming from - but we have 2 competing goals - "work with as many existing, external build tools as possible" versus "take this opportunity to create a new cross-compile-capable x64 build environment for Windows and let those external tools deal with the breakage". As I mentioned in a previous email, my personal opinion would be swayed by looking externally. Specifically, if we could determine the likelihood of external build processes (eg, mozilla) working unchanged if we stick with 'PCBuild', and if we could determine the cross-compilation strategy being adopted by the external libs we use (zlib, bsddb, etc), I think we could make an informed decision. > > You might be; I will be sad. It comes for a price, > I well understand the benefits, I use it all the time, but the price > still eludes me. how can a different name for the output folder > for a different platform be such a big problem? Please see above - its not a problem if you think of the PCBuild8 process as the "last step" in a build process - but often it is not. External tools that use Python (ie, things you try and build after the Python build has completed) are impacted. I understand that you might not use such tools, but they do exist. > > Why should there be versioning problems with python25.dll? Are there > > any past issues with incompatibilities with any > python2x.dll release? > Someone could replace the python25.dll that we ship with > their own patched > version, thereby gaining backdoor access to the software. The way > windows searches for old style dlls makes this easy. Using the SxS > signed loading scheme, you can protect yourself up to a point > from such attacks. I'm not sure I buy this. If someone has enough access to your machine to change pythonxx.dll, you are pretty screwed already. > > What about the registry? > I don't know about the registry, what is it used for? Please see PC/getpathp.c in Python source tree. However, I agree that there are a number of things we could do to help Python play nicely on Vista. It might help if we can enumerate the specific problems and potential solutions in a more formal way (eg, a Python bug) Cheers, Mark -------------- next part -------------- A non-text attachment was scrubbed... Name: winmail.dat Type: application/ms-tnef Size: 3360 bytes Desc: not available Url : http://mail.python.org/pipermail/distutils-sig/attachments/20070524/c25f579d/attachment.bin From trentm at activestate.com Thu May 24 00:53:53 2007 From: trentm at activestate.com (Trent Mick) Date: Wed, 23 May 2007 15:53:53 -0700 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <4654C40D.1090803@v.loewis.de> References: <00af01c79b6f$2f863c30$1f0a0a0a@enfoldsystems.local> <46520F4F.5010502@v.loewis.de> <46549812.9080400@activestate.com> <4654C40D.1090803@v.loewis.de> Message-ID: <4654C601.7070000@activestate.com> Martin v. L?wis wrote: >> I use this patch in ActivePython to get distutils to find the correct >> PCbuild dir (see attached). > > Would you like to commit this to 2.6? (or perhaps 2.5 even?) Sure, if others think it is a good thing. Will do tomorrow unless I hear a -1 before then. Trent -- Trent Mick trentm at activestate.com From mhammond at skippinet.com.au Thu May 24 01:12:40 2007 From: mhammond at skippinet.com.au (Mark Hammond) Date: Thu, 24 May 2007 09:12:40 +1000 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <4654C601.7070000@activestate.com> Message-ID: <032601c79d8f$dc7c8260$1f0a0a0a@enfoldsystems.local> > Martin v. L?wis wrote: > >> I use this patch in ActivePython to get distutils to find > the correct > >> PCbuild dir (see attached). > > > > Would you like to commit this to 2.6? (or perhaps 2.5 even?) > > Sure, if others think it is a good thing. Will do tomorrow > unless I hear > a -1 before then. I'm not quite a '-1', but am a little confused about where this would leave us. To some extent, this would formalize PCBuild8 and VC6 directories. External tools would then slowly start growing support for these additional directories and the previous benefits of "PCBuild is the canonical location" appear to vanish. Further, I expect that such a patch would confuse any attempts to manually copy from PCBuild8 into PCBuild, for example (ie, some tools knowing about PCBuild8 while others assume PCBuild is likely to get confusing.) Trent: I assume you use the same source tree for multiple platforms and compilers, meaning that changing these "optional" build processes to copy from PCBuild8/VC6 into PCBuild would cause pain? If not, do you think that would be a reasonable solution? Cheers, Mark From trentm at activestate.com Thu May 24 01:29:20 2007 From: trentm at activestate.com (Trent Mick) Date: Wed, 23 May 2007 16:29:20 -0700 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <032601c79d8f$dc7c8260$1f0a0a0a@enfoldsystems.local> References: <032601c79d8f$dc7c8260$1f0a0a0a@enfoldsystems.local> Message-ID: <4654CE50.2020004@activestate.com> > I'm not quite a '-1', but am a little confused about where this would leave > us. To some extent, this would formalize PCBuild8 and VC6 directories. > External tools would then slowly start growing support for these additional > directories and the previous benefits of "PCBuild is the canonical location" > appear to vanish. Further, I expect that such a patch would confuse any > attempts to manually copy from PCBuild8 into PCBuild, for example (ie, some > tools knowing about PCBuild8 while others assume PCBuild is likely to get > confusing.) > > Trent: I assume you use the same source tree for multiple platforms and > compilers, meaning that changing these "optional" build processes to copy > from PCBuild8/VC6 into PCBuild would cause pain? If not, do you think that > would be a reasonable solution? Changing to have bits always in PCbuild would work for me -- i.e. I *don't* build for multiple compilers/platforms in the same tree. Perhaps that is a better solution -- in the long run, anyway. Having the "bits" always in one dir for whatever the configuration is more akin to the Unix-y configure/make system. Is this something that could work for Python 2.5? Or just 2.6? Long term/aside: Moving to a configure/make build system on Windows, as you proposed in your first email, would be interesting. With MSYS though, not cygwin (a la bsmedberg's new MozillaBuild stuff). I just wish there were an autoconf alternative that wasn't as painful as autoconf. I have a few attempts for my purposes that are written in Python (an obvious bootstrapping problem for building Python itself :). Trent -- Trent Mick trentm at activestate.com From kristjan at ccpgames.com Thu May 24 12:58:05 2007 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=) Date: Thu, 24 May 2007 10:58:05 +0000 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <4654CE50.2020004@activestate.com> References: <032601c79d8f$dc7c8260$1f0a0a0a@enfoldsystems.local> <4654CE50.2020004@activestate.com> Message-ID: <4E9372E6B2234D4F859320D896059A9508DCBEEBCD@exchis.ccp.ad.local> > -----Original Message----- > though, not cygwin (a la bsmedberg's new MozillaBuild stuff). I just > wish there were an autoconf alternative that wasn't as painful as > autoconf. I have a few attempts for my purposes that are written in > Python (an obvious bootstrapping problem for building Python itself :). > Only for the theorist. As you, we use build tools for our stackless branch written in python. There exist successfully built python versions back from the nineties, which can be considered "external" tools for all practical purposes. Building python with python is really nifty. Kristjan From fairwinds at eastlink.ca Fri May 25 07:58:56 2007 From: fairwinds at eastlink.ca (David Pratt) Date: Fri, 25 May 2007 02:58:56 -0300 Subject: [Distutils] Eggs not picking up some files Message-ID: <46567B20.3090705@eastlink.ca> I've got sort of a strange issue with eggs for distribution to pypi. I am using the following command to create a egg and send it to pypi and it is omitting my .zcml files from the egg somehow. I export the source from svn and then run: python setup.py register bdist_egg sdist upload I thought maybe there was some sort of issue on my platform. I am on MacOSX 10.4.9 so tried from on of my FreeBSD servers and getting same thing. Probably something simple but hoping someone has an explanation. Many thanks. Regards, David From r.ritz at biologie.hu-berlin.de Fri May 25 09:30:29 2007 From: r.ritz at biologie.hu-berlin.de (Raphael Ritz) Date: Fri, 25 May 2007 09:30:29 +0200 Subject: [Distutils] Eggs not picking up some files In-Reply-To: <46567B20.3090705@eastlink.ca> References: <46567B20.3090705@eastlink.ca> Message-ID: David Pratt schrieb: > I've got sort of a strange issue with eggs for distribution to pypi. I > am using the following command to create a egg and send it to pypi and > it is omitting my .zcml files from the egg somehow. > > I export the source from svn and then run: > > python setup.py register bdist_egg sdist upload > > I thought maybe there was some sort of issue on my platform. I am on > MacOSX 10.4.9 so tried from on of my FreeBSD servers and getting same > thing. Probably something simple but hoping someone has an explanation. > Many thanks. > Maybe you post your setup.py here as well? Or in case you have not seen this yet http://peak.telecommunity.com/DevCenter/setuptools#including-data-files may already provide an answer. Raphael > Regards, > David > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > From fairwinds at eastlink.ca Fri May 25 14:48:12 2007 From: fairwinds at eastlink.ca (David Pratt) Date: Fri, 25 May 2007 09:48:12 -0300 Subject: [Distutils] Eggs not picking up some files In-Reply-To: References: <46567B20.3090705@eastlink.ca> Message-ID: <4656DB0C.3080606@eastlink.ca> Hi Raphael. Indeed, I exported my sources instead of checking them out of svn so it did not add the files to the egg since they were no longer under version control. Many thanks for pointing this out. Regards, David Raphael Ritz wrote: > David Pratt schrieb: >> I've got sort of a strange issue with eggs for distribution to pypi. I >> am using the following command to create a egg and send it to pypi and >> it is omitting my .zcml files from the egg somehow. >> >> I export the source from svn and then run: >> >> python setup.py register bdist_egg sdist upload >> >> I thought maybe there was some sort of issue on my platform. I am on >> MacOSX 10.4.9 so tried from on of my FreeBSD servers and getting same >> thing. Probably something simple but hoping someone has an explanation. >> Many thanks. >> > > Maybe you post your setup.py here as well? > > Or in case you have not seen this yet > > http://peak.telecommunity.com/DevCenter/setuptools#including-data-files > > may already provide an answer. > > Raphael > > >> Regards, >> David >> _______________________________________________ >> Distutils-SIG maillist - Distutils-SIG at python.org >> http://mail.python.org/mailman/listinfo/distutils-sig >> > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > From kristjan at ccpgames.com Sat May 26 14:01:10 2007 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=) Date: Sat, 26 May 2007 12:01:10 +0000 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <4654C15F.2040906@v.loewis.de> References: <009f01c79b51$b1da52c0$1f0a0a0a@enfoldsystems.local> <46512B09.1080304@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE4B0@exchis.ccp.ad.local> <4652128B.3000301@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE6FB@exchis.ccp.ad.local> <465365DE.1090306@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEEACB@exchis.ccp.ad.local> <4654C15F.2040906@v.loewis.de> Message-ID: <4E9372E6B2234D4F859320D896059A9508DE8B403C@exchis.ccp.ad.local> > -----Original Message----- > From: "Martin v. L?wis" [mailto:martin at v.loewis.de] > > It doesn't need to, and reluctance is not wrt. to the proposed new > layout, but wrt. changing the current one. Tons of infrastructure > depends on the files having exactly the names that they have now, > and being located in exactly the locations where they are currently > located. Any change to that, whatever minor, will cause problems > to some people. Just to be absolutely clear: You are talking about the build environment, right? Because I am not proposing to change any layout of the installed Python (wherever that may be :) I am baffled about why the build environment's layout matters, but once an .msi install can place the binaries in any old place it wants. The build structure doesn't have to reflect the final installed structure at all. Kristj?n From kristjan at ccpgames.com Sat May 26 14:20:59 2007 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=) Date: Sat, 26 May 2007 12:20:59 +0000 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: References: <009f01c79b51$b1da52c0$1f0a0a0a@enfoldsystems.local> <46512B09.1080304@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE4B0@exchis.ccp.ad.local> <4652128B.3000301@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE6FB@exchis.ccp.ad.local> <465365DE.1090306@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEEACB@exchis.ccp.ad.local> Message-ID: <4E9372E6B2234D4F859320D896059A9508DE8B4046@exchis.ccp.ad.local> > -----Original Message----- > From: Alexey Borzenkov [mailto:snaury at gmail.com] > Sent: Wednesday, May 23, 2007 20:36 > To: Kristj?n Valur J?nsson > Cc: Martin v. L?wis; Mark Hammond; distutils-sig at python.org; python- > dev at python.org > Subject: Re: [Python-Dev] Adventures with x64, VS7 and VS8 on Windows > > On 5/23/07, Kristj?n Valur J?nsson wrote: > > > > Install in the ProgramFiles folder. > > > Only over my dead body. *This* is silly. > > Bill doesn't think so. And he gets to decide. I mean we do want > > to play nice, don't we? Nothing installs itself in the root anymore, > > not since windows 3.1 > > Maybe installing in the root is not good, but installing to "Program > Files" is just asking for trouble. All sorts of development tools > might suddenly break because of that space in the middle of the path > and requirement to use quotes around it. I thus usually install things > to :\Programs. I'm not sure if any packages/programs will break > because of that space, but what if some will? Development tools used on windows already have to cope with this. Spaces are not going away, so why not bite the bullet and deal with them? Moving forward sometimes means crossing rivers. As for Vista issues, I'll gather more data before making any more claims, but I think that it is important that we play by the rules here. Just imagine the a school teacher who in good faith wants to introduce his pupils to the wonderful programming language of Python, but when he installs it, all kinds of scary looking warnings drive him off. Vista is, like it or not, going to be very prevalent. If we want python to be easily accessible to the masses, we mustn't take an elitist attitude or else risk scaring people off. Finally, to add a (mis)quote from Mr. Gorbachev: "Wer zu sp?t kommt, den bestraft das Leben" Kristj?n From kristjan at ccpgames.com Sat May 26 15:31:09 2007 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=) Date: Sat, 26 May 2007 13:31:09 +0000 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <4654C15F.2040906@v.loewis.de> References: <009f01c79b51$b1da52c0$1f0a0a0a@enfoldsystems.local> <46512B09.1080304@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE4B0@exchis.ccp.ad.local> <4652128B.3000301@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE6FB@exchis.ccp.ad.local> <465365DE.1090306@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEEACB@exchis.ccp.ad.local> <4654C15F.2040906@v.loewis.de> Message-ID: <4E9372E6B2234D4F859320D896059A9508DE8B405A@exchis.ccp.ad.local> > -----Original Message----- > > I personally think that if hostile users can replace DLLs on your > system, you have bigger problems than SxS installation of > pythhonxy.dll, > but perhaps that's just me. An end user application on an end-user's machine is always voulnerable to reverse engineering. But it helps to make it more difficult. The old-style dll loader is a particular vulnerability which makes it easy to patch into an application at load time. > >> What about the registry? > > I don't know about the registry, what is it used for? > > For two things, with different importance to different users: > 1. File extensions are registered there, e.g. .py and .pyc. > With two binaries installed, the will stomp over each other's > file associations; only one of them can win. Sure. No argument about this. But as with the explorer and other apps, it is perfectly possible to manually start one or the other, autoclicking on .py files isn't the only option. > 2. Python installs keys under > HL{LM|CU}\Software\Python\PythonCore\, namely > InstallPath > InstallGroup > PythonPath > Documentation > Modules > Funnily enough, Bill has thought of this too. See http://support.microsoft.com/kb/305097/ for info. Co-existence of 32 and 64 bit apps is supported. > > The two versions of MSIE actually *are* a big problem, that's > why MS only runs the 32-bit IE, even on Win64 (otherwise, ActiveX > controls downloaded from the net wouldn't work). > > Also, while they are both shipped, you can't the two versions of > explorer.exe simultaneously (without trickery), so its far from > simple. The two versions aren't the problem, it is the backward support for 32 bit active thingies that are, as you point out. There is confusion here: internet explorer shipped in both versions. The 32 bit version war default for the above reason. But explorer.exe (which I was talking about) also had two versions. The 64 bit version ran by default. You may recall that before tortoise shipped a 64 bit version, one had to kill explorer.exe and restart it (explorer32.exe IIRC) to get tortoise to work. Supporting both kinds (country and western) on the same machine might be helpful to people for this very reason. A lot of legacy modules are only avaible in 32 bit mode. But people may want to do contemporary development using the new 64 bit mode. Cheers, Kristj?n From p_thomas at gmx.net Sun May 27 12:22:10 2007 From: p_thomas at gmx.net (Thomas) Date: Sun, 27 May 2007 12:22:10 +0200 Subject: [Distutils] distutils.core.setup: ImportError: undefined symbol: g_free Message-ID: <1180261330.742.9.camel@skynet> Hallo, I want to call some C functions from the gnome-panel source. setup.py: > setup(name = "calwrap", version = "1.0", ext_modules = [ ... Then I call: ./setup.py build Compiling works fine, I get: calwrap.so Then in python I get the following error: > Python 2.5.1 (r251:54863, May 2 2007, 16:56:35) > [GCC 4.1.2 (Ubuntu 4.1.2-0ubuntu4)] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > >>> import calwrap > Traceback (most recent call last): > File "", line 1, in > ImportError: ./calwrap.so: undefined symbol: g_free I tried: > extra_link_args = ["-Xlinker", "-export-dynamic"] Any ideas? fg Thomas From martin at v.loewis.de Tue May 29 00:09:04 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 29 May 2007 00:09:04 +0200 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <4E9372E6B2234D4F859320D896059A9508DE8B403C@exchis.ccp.ad.local> References: <009f01c79b51$b1da52c0$1f0a0a0a@enfoldsystems.local> <46512B09.1080304@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE4B0@exchis.ccp.ad.local> <4652128B.3000301@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE6FB@exchis.ccp.ad.local> <465365DE.1090306@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEEACB@exchis.ccp.ad.local> <4654C15F.2040906@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DE8B403C@exchis.ccp.ad.local> Message-ID: <465B5300.8090901@v.loewis.de> >> It doesn't need to, and reluctance is not wrt. to the proposed new >> layout, but wrt. changing the current one. Tons of infrastructure >> depends on the files having exactly the names that they have now, >> and being located in exactly the locations where they are currently >> located. Any change to that, whatever minor, will cause problems >> to some people. > > Just to be absolutely clear: You are talking about the build environment, > right? Because I am not proposing to change any layout of the > installed Python (wherever that may be :) Correct. > I am baffled about why the build environment's layout matters, > but once an .msi install can place the binaries in any > old place it wants. The build structure doesn't have to > reflect the final installed structure at all. No. But still, people like to be able to "run" Python out of a source check-out. This has been supported for a long time, and more and more stuff was added to support it. For examples within Python itself, see the support in distutils, getpathp.c, PCbuild/rt.bat, and Tools/buildbot/*.bat. Reportedly (by Mark), building Mozilla (the web browser) also "knows" about PCbuild. Regards, Martin From martin at v.loewis.de Tue May 29 00:13:03 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 29 May 2007 00:13:03 +0200 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <4E9372E6B2234D4F859320D896059A9508DE8B4046@exchis.ccp.ad.local> References: <009f01c79b51$b1da52c0$1f0a0a0a@enfoldsystems.local> <46512B09.1080304@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE4B0@exchis.ccp.ad.local> <4652128B.3000301@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE6FB@exchis.ccp.ad.local> <465365DE.1090306@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEEACB@exchis.ccp.ad.local> <4E9372E6B2234D4F859320D896059A9508DE8B4046@exchis.ccp.ad.local> Message-ID: <465B53EF.6070108@v.loewis.de> > Development tools used on windows already have to cope with this. > Spaces are not going away, so why not bite the bullet and deal > with them? Moving forward sometimes means crossing rivers. But in a safe path, step by step. People continue to report problems with spaces in file names, even though many of them have been solved also. > As for Vista issues, I'll gather more data before making any more > claims, but I think that it is important that we play by the rules > here. I completely disagree. It is important that it "works", not that we play by the rules. > Just imagine the a school teacher who in good faith wants to introduce > his pupils to the wonderful programming language of Python, but > when he installs it, all kinds of scary looking warnings drive him off. > Vista is, like it or not, going to be very prevalent. If we want python > to be easily accessible to the masses, we mustn't take an elitist > attitude or else risk scaring people off. I'm completely in favor of fixing actual bugs. However, I'm not aware of any (related to Vista). That it is not logo compliant is *not* a bug. Python hasn't been logo compliant for more than a decade now (the "install to Program Files" is not a new requirement, but existed since Win95). Regards, Martin From martin at v.loewis.de Tue May 29 00:16:59 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 29 May 2007 00:16:59 +0200 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <4E9372E6B2234D4F859320D896059A9508DE8B405A@exchis.ccp.ad.local> References: <009f01c79b51$b1da52c0$1f0a0a0a@enfoldsystems.local> <46512B09.1080304@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE4B0@exchis.ccp.ad.local> <4652128B.3000301@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE6FB@exchis.ccp.ad.local> <465365DE.1090306@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEEACB@exchis.ccp.ad.local> <4654C15F.2040906@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DE8B405A@exchis.ccp.ad.local> Message-ID: <465B54DB.3040809@v.loewis.de> > Supporting both kinds (country and western) on the same machine might be helpful > to people for this very reason. A lot of legacy modules are only avaible > in 32 bit mode. But people may want to do contemporary development using the > new 64 bit mode. Of course, people who really want that can install both versions simultaneously today. Regards, Martin From mhammond at skippinet.com.au Tue May 29 05:11:33 2007 From: mhammond at skippinet.com.au (Mark Hammond) Date: Tue, 29 May 2007 13:11:33 +1000 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: Message-ID: <03d801c7a19f$1033f120$1f0a0a0a@enfoldsystems.local> > From: Jamie Kirkpatrick [mailto:jkp at kirkconsulting.co.uk] > Sent: Wednesday, 23 May 2007 5:16 AM > I have a set of extensions that use SWIG to wrap my own C++ library. This library, on a > day-to-day basis is built against VS8 since the rest of our product suite is. Right now > I have no way to work with this code using VS8 since the standard distribution is built > against VS7 which uses a different CRT. This is an absolute nightmare in practice since > I currently have to maintain VS7 projects in parallel with the standard VS8 ones simply > so that I can run and test my python code. If you are brave and willing to ensure your module doesn't voilate certain constraints (such as never passing a CRT 'concept' - such as a file handle or memory block to be free'd) across mismatched CRT boundaries, you may find that you can happily load your VS8 built pronect with VS7 - but yes, your general point is valid but beyond the scope of this discussion. A separate discussion on making Python "crt agnostic" is almost certainly worthwhile though, but not directly related to this current discussion. > I've downloaded the Python source and had a look at building up my own distributions for each case > (ideally there would be an easy way to separate out Release / Debug products as well as the > VS8 / VS7 variants, and I guess potentially for those cross-compiling we'd need to go a step > further and do this per arch as well. Anyway, this isn't how it works at the moment, but I'm > still searching for a way to be able to work on the python code in VS8. Building using the > current projects I seem to get everything in the PCBuild8 / PCBuild dirs. How can I work with > what is build? This is *exactly* the point of this thread, and what we are trying to resolve. In the short term, we have agreed a change to PCBuild8\build.bat that copies the build files into PCBuild is a solution that should "work", where "work" is defined as "allow a source tree built with VS8 to operate in the same way, from the POV of building extensions, as one built with VS8." My primary issue with this is solved by the change to the .bat file, but we welcome all feedback from people who believe this is not ideal. I've agreement from Kristjan on the specific change to that .bat file, I'm just yet to check it in (but it literally just copies everything from the PVBuild8 target dir into the PCBuild dir after checking the expected dirs do indeed exist) > Is there a shell script to build a final distribution tree? If not, is there a simple > way to build an MSI similar to the one found on the Python.org site for the official > releases but using the PCBuild8 stuff? I believe not. In most cases, people who build from source on Windows will run directly from that source tree, rather than attempting the intermediate step of creating a .msi and installing it. > If not how do you recommend getting myself to a state where I have at least a feature complete > distribution build against VS8? I'm happy with a one time build that I can just install into > my source tree and upload to the SCM. I'd suggest that once I check the .bat change in, you build the PCBuild8 directory via that .bat file, then continue to use the 'PCBuild' directory as it it were a VC7 build. Cheers, Mark From martin at v.loewis.de Tue May 29 06:27:17 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 29 May 2007 06:27:17 +0200 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <03d801c7a19f$1033f120$1f0a0a0a@enfoldsystems.local> References: <03d801c7a19f$1033f120$1f0a0a0a@enfoldsystems.local> Message-ID: <465BABA5.5030301@v.loewis.de> >> Is there a shell script to build a final distribution tree? If >> not, is there a simple way to build an MSI similar to the one found >> on the Python.org site for the official releases but using the >> PCBuild8 stuff? > > I believe not. It's actually not that difficult. You just have to run Tools/msi/msi.py, with a python interpreter that support PythonCOM (i.e. has PythonWin installed). Replacing all occurrences of PC[Bb]uild with PCbuild8 should do the trick. This will give you a "snapshot" MSI, i.e. one that you can install along with a regularly-release Python interpreter. If you put a config.py with testpackage=True into the msi directory, it will register the extensions .px, .pxw, .pxo, .pxc (rather than .py*), and overwrite the registry at Software\xPython (rather than Software\Python), so that the package shouldn't interfere at all with a regular installation. Regards, Martin From ncoghlan at gmail.com Tue May 29 15:27:36 2007 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 29 May 2007 23:27:36 +1000 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <46536787.7040500@v.loewis.de> References: <00af01c79b6f$2f863c30$1f0a0a0a@enfoldsystems.local> <46520F4F.5010502@v.loewis.de> <46536787.7040500@v.loewis.de> Message-ID: <465C2A48.3030205@gmail.com> Martin v. L?wis wrote: >> I have a set of extensions that use SWIG to wrap my own C++ library. >> This library, on a day-to-day basis is built against VS8 since the rest >> of our product suite is. Right now I have no way to work with this code >> using VS8 since the standard distribution is built against VS7 which >> uses a different CRT. This is an absolute nightmare in practice since >> I currently have to maintain VS7 projects in parallel with the standard >> VS8 ones simply so that I can run and test my python code. > > If you know well enough what you are doing, and avoid using unsupported > cases, you *can* mix different CRTs. I can attest to this - I have SWIG-wrapped extensions built with VC6 running quite happily against the official VS7 binaries for Python 2.4. Moving from Python 2.2 to Python 2.4 was a simple matter of recompiling and relinking the modules. The important thing was to make sure to never pass memory ownership or standard lib data structures across the boundary. I haven't actually found this to be all that difficult in practice, as I am typically either copying data from standard library data structures into native Python data structures (e.g. between std::string and PyString) or else merely accessing the Python wrappers around my own C++ classes. In both cases memory ownership remains entirely within the extension module, and all interaction occurs through the Python C API, and never indirectly through the CRT. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From kristjan at ccpgames.com Tue May 29 15:29:58 2007 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=) Date: Tue, 29 May 2007 13:29:58 +0000 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <465B53EF.6070108@v.loewis.de> References: <009f01c79b51$b1da52c0$1f0a0a0a@enfoldsystems.local> <46512B09.1080304@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE4B0@exchis.ccp.ad.local> <4652128B.3000301@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE6FB@exchis.ccp.ad.local> <465365DE.1090306@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEEACB@exchis.ccp.ad.local> <4E9372E6B2234D4F859320D896059A9508DE8B4046@exchis.ccp.ad.local> <465B53EF.6070108@v.loewis.de> Message-ID: <4E9372E6B2234D4F859320D896059A9508DE8B435C@exchis.ccp.ad.local> > -----Original Message----- > From: "Martin v. L?wis" [mailto:martin at v.loewis.de] > > Just imagine the a school teacher who in good faith wants to > introduce > > his pupils to the wonderful programming language of Python, but > > when he installs it, all kinds of scary looking warnings drive him > off. > > Vista is, like it or not, going to be very prevalent. If we want > python > > to be easily accessible to the masses, we mustn't take an elitist > > attitude or else risk scaring people off. > > I'm completely in favor of fixing actual bugs. However, I'm not aware > of any (related to Vista). That it is not logo compliant is *not* > a bug. Python hasn't been logo compliant for more than a decade > now (the "install to Program Files" is not a new requirement, but > existed since Win95). > I'm not saying that it is a bug, but since this is python-dev, I am discussing it as a desirable "feature". One feature that is easily addable and will certainly make installing python on vista nicer, is to add authenticode signing to the install. Currently the user is faced with a very nasty and off-putting message about an unidentified program requesting access to his computer. See http://msdn2.microsoft.com/en-us/library/bb172338.aspx . I think the PSF should be able to obtain a certificate from MS. cheers, Kristj?n From sklein at cpcug.org Tue May 29 20:10:47 2007 From: sklein at cpcug.org (Stanley A. Klein) Date: Tue, 29 May 2007 14:10:47 -0400 (EDT) Subject: [Distutils] I need to create a customized Fedora live cd with Python applications on it In-Reply-To: <20070522220044.C82EA3A4061@sparrow.telecommunity.com> References: <24685.207.188.248.157.1179867688.squirrel@www.cpcug.org> <20070522210151.AB6BE3A4061@sparrow.telecommunity.com> <38000.207.188.248.157.1179869031.squirrel@www.cpcug.org> <20070522220044.C82EA3A4061@sparrow.telecommunity.com> Message-ID: <31168.207.188.248.157.1180462247.squirrel@www.cpcug.org> I found a report on bugzilla.redhat.com, "Bug 236535 Processed: brp-python-bytecompile does not work with bdist-rpm target" and am cc:'ing the cc: list for that bug on this email. I posted a comment to that bug strongly urging the Fedora packaging people and the Python packaging people to get their heads together and resolve this issue. For the benefit of the people on the Fedora bug cc: list, a posting to the Python distutils list that describes my problem can be found at http://mail.python.org/pipermail/distutils-sig/2007-May/007639.html My previous message with the attached entire output of bdist_rpm for ZSI-2.0 and ZSI-2.0-rc3 was too large for posting to the list. I've provided the remainder of the message below, which raises yet another issue involving coordination of dependency processing. I hope that by putting the major players together by email you can resolve the bdist_rpm issue soon. Thanks. Stan Klein On Tue, 2007-05-22 at 18:02 -0400, Phillip J. Eby wrote: > At 05:23 PM 5/22/2007 -0400, Stanley A. Klein wrote: > >Prior to attempting the rpm build, I updated my version of setuptools to > >0.6c5 (package python-setuptools-0.6c5-1.fc5). > Would you mind emailing me the *entire* output of the bdist_rpm operation? When I try this with 0.6c5 and the source of ZSI-2.0-rc3, I don't get the errors (although I do see the "files listed twice" warnings). Thanks. I've attached below the entire output of the bdist_rpm operations for both ZSI-2.0 and ZSI-2.0-rc3. The same errors occurred for both cases. Also, I verified that I have the latest version in the FC5 repository of the rpm-build package, in case the issue originates in that package. My having to re-untar the rc3 reminded me of another problem I've had running setup.py for this package. ZSI has a dependency on PyXML expressed as additional_params['install_requires'] = [ "PyXML >= 0.8.3", ] I have the proper version of PyXML installed on my systems, but via an RPM. Setuptools consistently fails to recognize that a proper version of PyXML is there, and unless I comment out the dependency statement it tries to find and download the latest version of PyXML from (apparently incorrect) information in setup.py. ZSI can find PyXML, but apparently setup.py can't. Stan Klein [Attachment removed because it is too large for the email list.] -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20070529/0b81ff4c/attachment.htm From pje at telecommunity.com Tue May 29 21:43:37 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 29 May 2007 15:43:37 -0400 Subject: [Distutils] I need to create a customized Fedora live cd with Python applications on it In-Reply-To: <31168.207.188.248.157.1180462247.squirrel@www.cpcug.org> References: <24685.207.188.248.157.1179867688.squirrel@www.cpcug.org> <20070522210151.AB6BE3A4061@sparrow.telecommunity.com> <38000.207.188.248.157.1179869031.squirrel@www.cpcug.org> <20070522220044.C82EA3A4061@sparrow.telecommunity.com> <31168.207.188.248.157.1180462247.squirrel@www.cpcug.org> Message-ID: <20070529194217.CEC093A409D@sparrow.telecommunity.com> At 02:10 PM 5/29/2007 -0400, Stanley A. Klein wrote: >I found a report on bugzilla.redhat.com, "Bug 236535 Processed: >brp-python-bytecompile does not work with bdist-rpm target" and am cc:'ing >the cc: list for that bug on this email. > >I posted a comment to that bug strongly urging the Fedora packaging people >and the Python packaging people to get their heads together and resolve >this issue. > >For the benefit of the people on the Fedora bug cc: list, a posting to the >Python distutils list that describes my problem can be found at >http://mail.python.org/pipermail/distutils-sig/2007-May/007639.html Hi Stanley. I reviewed the notes on the bug; this definitely explains why I haven't been able to reproduce your problem under CentOS (which is where I was testing it). It looks to me like there are two possible long-term fixes: 1. The Fedora folks could fix brp-python-bytecompile to automatically add the *.pyo files to the installed-files list (and adjust the compilation location) to remove the error(s). (I think this is the best approach, as it ensures Fedora's policy is followed even for user-built RPMs.) 2. I could change setuptools to always generate *.pyo files when creating RPMs, or to have the option to do so -- and we'd also have to change the distutils in Python. The big problem with option #2 is that even if it were considered a bugfix for Python (which it *isn't*, since clearly the Fedora policy change is what's breaking it), it would have to wait until a Python 2.5.x release. Is Fedora even *using* 2.5 yet? In the meantime, there is one workaround you can use with your distutils package. Add the following two lines to 'setup.cfg' in the package root, adjacent to the 'setup.py' file: [install] optimize = 1 This will force all RPM's built with the package to include *.pyo files, which should then dodge the (By the way, *.pyo files actually aren't all that useless. More recent versions of Python actually do a few optimizations on the generated bytecode, including things like constant folding, in addition to removing "if __debug__:" blocks and "assert" statements.) From sklein at cpcug.org Tue May 29 22:20:30 2007 From: sklein at cpcug.org (Stanley A. Klein) Date: Tue, 29 May 2007 16:20:30 -0400 (EDT) Subject: [Distutils] I need to create a customized Fedora live cd with Python applications on it In-Reply-To: <20070529194217.CEC093A409D@sparrow.telecommunity.com> References: <24685.207.188.248.157.1179867688.squirrel@www.cpcug.org> <20070522210151.AB6BE3A4061@sparrow.telecommunity.com> <38000.207.188.248.157.1179869031.squirrel@www.cpcug.org> <20070522220044.C82EA3A4061@sparrow.telecommunity.com> <31168.207.188.248.157.1180462247.squirrel@www.cpcug.org> <20070529194217.CEC093A409D@sparrow.telecommunity.com> Message-ID: <31648.207.188.248.157.1180470030.squirrel@www.cpcug.org> On Tue, May 29, 2007 3:43 pm, Phillip J. Eby wrote: > At 02:10 PM 5/29/2007 -0400, Stanley A. Klein wrote: >>I found a report on bugzilla.redhat.com, "Bug 236535 Processed: >>brp-python-bytecompile does not work with bdist-rpm target" and am >> cc:'ing >>the cc: list for that bug on this email. >> >>I posted a comment to that bug strongly urging the Fedora packaging >> people >>and the Python packaging people to get their heads together and resolve >>this issue. >> >>For the benefit of the people on the Fedora bug cc: list, a posting to >> the >>Python distutils list that describes my problem can be found at >>http://mail.python.org/pipermail/distutils-sig/2007-May/007639.html > > Hi Stanley. I reviewed the notes on the bug; this definitely > explains why I haven't been able to reproduce your problem under > CentOS (which is where I was testing it). > > It looks to me like there are two possible long-term fixes: > > 1. The Fedora folks could fix brp-python-bytecompile to automatically > add the *.pyo files to the installed-files list (and adjust the > compilation location) to remove the error(s). (I think this is the > best approach, as it ensures Fedora's policy is followed even for > user-built RPMs.) > > 2. I could change setuptools to always generate *.pyo files when > creating RPMs, or to have the option to do so -- and we'd also have > to change the distutils in Python. > > The big problem with option #2 is that even if it were considered a > bugfix for Python (which it *isn't*, since clearly the Fedora policy > change is what's breaking it), it would have to wait until a Python > 2.5.x release. Is Fedora even *using* 2.5 yet? > > In the meantime, there is one workaround you can use with your > distutils package. Add the following two lines to 'setup.cfg' in the > package root, adjacent to the 'setup.py' file: > > [install] > optimize = 1 > > This will force all RPM's built with the package to include *.pyo > files, which should then dodge the > > (By the way, *.pyo files actually aren't all that useless. More > recent versions of Python actually do a few optimizations on the > generated bytecode, including things like constant folding, in > addition to removing "if __debug__:" blocks and "assert" statements.) > I used your workaround and got the rpms. However, I also got a traceback, attached below. I think this is another problem in setuptools, because (IIRC) I saw a posting relating to it. I hope we are on the way to a general solution. Fedora Core 5 (which I currently run pending release of Fedora 7) distributes setuptools as a separate package. If Fedora 7 continues that policy, all that might be needed is a new release of setuptools with an option (perhaps --Fedora) that have the same effect as putting the two lines in the setup.cfg file. Alternatively, the Fedora distribution could put the lines into some kind of global config file and setuptools could have an option (perhaps --noFedora) to not use them. Meanwhile, what is needed is to give much greater publicity to the workaround. Thanks. Stan Klein -------------------------------------------------------------------------- (The traceback is at the bottom, but I've included more above to show that the packaging seemed to work.) Installing wsdl2dispatch script to /var/tmp/ZSI-2.0-1-buildroot/usr/bin Installing wsdl2py script to /var/tmp/ZSI-2.0-1-buildroot/usr/bin writing list of installed files to 'INSTALLED_FILES' + /usr/lib/rpm/find-debuginfo.sh /home/stan/exper/zsi2/ZSI-2.0/build/bdist.linux-i686/rpm/BUILD/ZSI-2.0 0 blocks find: /var/tmp/ZSI-2.0-1-buildroot/usr/lib/debug: No such file or directory + /usr/lib/rpm/redhat/brp-compress + /usr/lib/rpm/redhat/brp-strip-static-archive /usr/bin/strip + /usr/lib/rpm/redhat/brp-strip-comment-note /usr/bin/strip /usr/bin/objdump + /usr/lib/rpm/brp-python-bytecompile Processing files: ZSI-2.0-1 Executing(%doc): /bin/sh -e /var/tmp/rpm-tmp.59999 + umask 022 + cd /home/stan/exper/zsi2/ZSI-2.0/build/bdist.linux-i686/rpm/BUILD + cd ZSI-2.0 + DOCDIR=/var/tmp/ZSI-2.0-1-buildroot/usr/share/doc/ZSI-2.0 + export DOCDIR + rm -rf /var/tmp/ZSI-2.0-1-buildroot/usr/share/doc/ZSI-2.0 + /bin/mkdir -p /var/tmp/ZSI-2.0-1-buildroot/usr/share/doc/ZSI-2.0 + cp -pr README CHANGES doc/zsi.css doc/zsi.html /var/tmp/ZSI-2.0-1-buildroot/usr/share/doc/ZSI-2.0 + exit 0 warning: File listed twice: /usr/lib/python2.4/site-packages/ZSI-2.0-py2.4.egg-info/PKG-INFO warning: File listed twice: /usr/lib/python2.4/site-packages/ZSI-2.0-py2.4.egg-info/SOURCES.txt warning: File listed twice: /usr/lib/python2.4/site-packages/ZSI-2.0-py2.4.egg-info/dependency_links.txt warning: File listed twice: /usr/lib/python2.4/site-packages/ZSI-2.0-py2.4.egg-info/entry_points.txt warning: File listed twice: /usr/lib/python2.4/site-packages/ZSI-2.0-py2.4.egg-info/requires.txt warning: File listed twice: /usr/lib/python2.4/site-packages/ZSI-2.0-py2.4.egg-info/top_level.txt Requires(rpmlib): rpmlib(CompressedFileNames) <= 3.0.4-1 rpmlib(PayloadFilesHavePrefix) <= 4.0-1 Requires: /usr/bin/python PyXML python(abi) = 2.4 Checking for unpackaged file(s): /usr/lib/rpm/check-files /var/tmp/ZSI-2.0-1-buildroot Wrote: /home/stan/exper/zsi2/ZSI-2.0/build/bdist.linux-i686/rpm/SRPMS/ZSI-2.0-1.src.rpm Wrote: /home/stan/exper/zsi2/ZSI-2.0/build/bdist.linux-i686/rpm/RPMS/noarch/ZSI-2.0-1.noarch.rpm Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.49538 + umask 022 + cd /home/stan/exper/zsi2/ZSI-2.0/build/bdist.linux-i686/rpm/BUILD + cd ZSI-2.0 + rm -rf /var/tmp/ZSI-2.0-1-buildroot + exit 0 Executing(--clean): /bin/sh -e /var/tmp/rpm-tmp.74009 + umask 022 + cd /home/stan/exper/zsi2/ZSI-2.0/build/bdist.linux-i686/rpm/BUILD + rm -rf ZSI-2.0 + exit 0 moving build/bdist.linux-i686/rpm/SRPMS/ZSI-2.0-1.src.rpm -> dist moving build/bdist.linux-i686/rpm/RPMS/noarch/ZSI-2.0-1.noarch.rpm -> dist Traceback (most recent call last): File "setup.py", line 66, in ? long_description="For additional information, please see " + _url, File "/usr/lib/python2.4/distutils/core.py", line 149, in setup dist.run_commands() File "/usr/lib/python2.4/distutils/dist.py", line 946, in run_commands self.run_command(cmd) File "/usr/lib/python2.4/distutils/dist.py", line 966, in run_command cmd_obj.run() File "/usr/lib/python2.4/site-packages/setuptools/command/bdist_rpm.py", line 28, in run _bdist_rpm.run(self) File "/usr/lib/python2.4/distutils/command/bdist_rpm.py", line 377, in run self.move_file(rpm, self.dist_dir) File "/usr/lib/python2.4/site-packages/setuptools/command/bdist_rpm.py", line 20, in move_file getattr(self.distribution,'dist_files',[]).append( NameError: global name 'get_python_version' is not defined From rudd-o at rudd-o.com Tue May 29 22:15:12 2007 From: rudd-o at rudd-o.com (Manuel Amador (Rudd-O)) Date: Tue, 29 May 2007 20:15:12 +0000 Subject: [Distutils] I need to create a customized Fedora live cd with Python applications on it In-Reply-To: <20070529194217.CEC093A409D@sparrow.telecommunity.com> References: <24685.207.188.248.157.1179867688.squirrel@www.cpcug.org> <20070522210151.AB6BE3A4061@sparrow.telecommunity.com> <38000.207.188.248.157.1179869031.squirrel@www.cpcug.org> <20070522220044.C82EA3A4061@sparrow.telecommunity.com> <31168.207.188.248.157.1180462247.squirrel@www.cpcug.org> <20070529194217.CEC093A409D@sparrow.telecommunity.com> Message-ID: <1180469712.4953.80.camel@gabriela> Funny how the solutions you advocated were the exact same solutions I advocated in the Fedora bugtracker months ago. I switched to KUbuntu. Thanks for the workaround tho. Please do NOT choose option 2, there's no reason to bend over to Fedora just because they're the second largest one, and there's no reason to litter filesystems in other distros with *.pyo files. Have a fantastic day, guys! On mar, 2007-05-29 at 15:43 -0400, Phillip J. Eby wrote: > At 02:10 PM 5/29/2007 -0400, Stanley A. Klein wrote: > >I found a report on bugzilla.redhat.com, "Bug 236535 Processed: > >brp-python-bytecompile does not work with bdist-rpm target" and am cc:'ing > >the cc: list for that bug on this email. > > > >I posted a comment to that bug strongly urging the Fedora packaging people > >and the Python packaging people to get their heads together and resolve > >this issue. > > > >For the benefit of the people on the Fedora bug cc: list, a posting to the > >Python distutils list that describes my problem can be found at > >http://mail.python.org/pipermail/distutils-sig/2007-May/007639.html > > Hi Stanley. I reviewed the notes on the bug; this definitely > explains why I haven't been able to reproduce your problem under > CentOS (which is where I was testing it). > > It looks to me like there are two possible long-term fixes: > > 1. The Fedora folks could fix brp-python-bytecompile to automatically > add the *.pyo files to the installed-files list (and adjust the > compilation location) to remove the error(s). (I think this is the > best approach, as it ensures Fedora's policy is followed even for > user-built RPMs.) > > 2. I could change setuptools to always generate *.pyo files when > creating RPMs, or to have the option to do so -- and we'd also have > to change the distutils in Python. > > The big problem with option #2 is that even if it were considered a > bugfix for Python (which it *isn't*, since clearly the Fedora policy > change is what's breaking it), it would have to wait until a Python > 2.5.x release. Is Fedora even *using* 2.5 yet? > > In the meantime, there is one workaround you can use with your > distutils package. Add the following two lines to 'setup.cfg' in the > package root, adjacent to the 'setup.py' file: > > [install] > optimize = 1 > > This will force all RPM's built with the package to include *.pyo > files, which should then dodge the > > (By the way, *.pyo files actually aren't all that useless. More > recent versions of Python actually do a few optimizations on the > generated bytecode, including things like constant folding, in > addition to removing "if __debug__:" blocks and "assert" statements.) > Manuel Amador (Rudd-O) The R Zone - http://rudd-o.com/ GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ Now playing, courtesy of Amarok: Proyecto Uno - The jam You will get what you deserve. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/distutils-sig/attachments/20070529/c0868498/attachment.pgp From pje at telecommunity.com Tue May 29 23:20:24 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 29 May 2007 17:20:24 -0400 Subject: [Distutils] I need to create a customized Fedora live cd with Python applications on it In-Reply-To: <31648.207.188.248.157.1180470030.squirrel@www.cpcug.org> References: <24685.207.188.248.157.1179867688.squirrel@www.cpcug.org> <20070522210151.AB6BE3A4061@sparrow.telecommunity.com> <38000.207.188.248.157.1179869031.squirrel@www.cpcug.org> <20070522220044.C82EA3A4061@sparrow.telecommunity.com> <31168.207.188.248.157.1180462247.squirrel@www.cpcug.org> <20070529194217.CEC093A409D@sparrow.telecommunity.com> <31648.207.188.248.157.1180470030.squirrel@www.cpcug.org> Message-ID: <20070529212504.665733A40A8@sparrow.telecommunity.com> At 04:20 PM 5/29/2007 -0400, Stanley A. Klein wrote: >I used your workaround and got the rpms. However, I also got a traceback, >attached below. I think this is another problem in setuptools, because >(IIRC) I saw a posting relating to it. I believe that was fixed in 0.6c5, if not, then in 0.6c6 (to be released Real Soon Now). >Alternatively, the Fedora distribution could put the lines into some kind >of global config file and setuptools could have an option (perhaps >--noFedora) to not use them. Good point - if they add those lines to the $prefix/lib/python2.x/distutils/distutils.cfg file, then all packages will install with .pyo files by default, thereby fixing the problem. This would be a fast fix for Fedora and ensure that all Python packages built and installed there (via distutils, setuptools, or bdist_rpm) will include .pyo files, unless specifically overridden. (By the way, you can always add those lines to the root user's $HOME/.pydistutils.cfg instead of editing individual setup.cfg files. Then, any packages installed (or RPMs built) by the root user will include .pyo files.) From martin at v.loewis.de Tue May 29 23:37:13 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 29 May 2007 23:37:13 +0200 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <4E9372E6B2234D4F859320D896059A9508DE8B435C@exchis.ccp.ad.local> References: <009f01c79b51$b1da52c0$1f0a0a0a@enfoldsystems.local> <46512B09.1080304@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE4B0@exchis.ccp.ad.local> <4652128B.3000301@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE6FB@exchis.ccp.ad.local> <465365DE.1090306@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEEACB@exchis.ccp.ad.local> <4E9372E6B2234D4F859320D896059A9508DE8B4046@exchis.ccp.ad.local> <465B53EF.6070108@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DE8B435C@exchis.ccp.ad.local> Message-ID: <465C9D09.9040401@v.loewis.de> > One feature that is easily addable and will certainly make installing > python on vista nicer, is to add authenticode signing to the install. This I question very much. I experimented with authenticode before 2.4, and found it an unacceptable experience. When the MSI file starts running, installer needs to verify the signature, for which it needs to compute a hash of the entire file. For the Python MSI, this takes many seconds on a slower Pentium 4 machine. During that time, there is no visual feedback, so users are uncertain whether they have actually invoked the MSI file at all. > Currently the user is faced with a very nasty and off-putting message > about an unidentified program requesting access to his computer. Certainly. However, telling them that they have to wait just so that Windows finds out what they know already (that this is the MSI file from the Python Software Foundation, or from Martin v. L?wis) is even more nasty. Regards, Martin From lars at ibp.de Wed May 30 00:22:31 2007 From: lars at ibp.de (Lars Immisch) Date: Wed, 30 May 2007 00:22:31 +0200 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <465C9D09.9040401@v.loewis.de> References: <009f01c79b51$b1da52c0$1f0a0a0a@enfoldsystems.local> <46512B09.1080304@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE4B0@exchis.ccp.ad.local> <4652128B.3000301@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE6FB@exchis.ccp.ad.local> <465365DE.1090306@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEEACB@exchis.ccp.ad.local> <4E9372E6B2234D4F859320D896059A9508DE8B4046@exchis.ccp.ad.local> <465B53EF.6070108@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DE8B435C@exchis.ccp.ad.local> <465C9D09.9040401@v.loewis.de> Message-ID: <465CA7A7.1070704@ibp.de> Hi, >> One feature that is easily addable and will certainly make installing >> python on vista nicer, is to add authenticode signing to the install. I'm +1 on authenticode. > This I question very much. I experimented with authenticode before 2.4, > and found it an unacceptable experience. When the MSI file starts > running, installer needs to verify the signature, for which it needs > to compute a hash of the entire file. For the Python MSI, this takes > many seconds on a slower Pentium 4 machine. During that time, there > is no visual feedback, so users are uncertain whether they have > actually invoked the MSI file at all. > >> Currently the user is faced with a very nasty and off-putting message >> about an unidentified program requesting access to his computer. > > Certainly. However, telling them that they have to wait just so that > Windows finds out what they know already (that this is the MSI file > from the Python Software Foundation, or from Martin v. L?wis) is > even more nasty. Educated, adult developers with good internet connections may know that, but all users? What about software on a CD or a memory stick? Also, software sites/mirrors have been compromised in the past, and they are a sweet target. I haven't looked at authenticode, but I guess it's a cryptographical signature. That defaults to a good thing. That the verification takes time is unfortunate, but unavoidable. That the user interface sucks (no feedback) is a bug. You will have the say whether Python uses authenticode, but I'm not convinced by your arguments. - Lars From lars at ibp.de Wed May 30 00:21:15 2007 From: lars at ibp.de (Lars Immisch) Date: Wed, 30 May 2007 00:21:15 +0200 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <465C9D09.9040401@v.loewis.de> References: <009f01c79b51$b1da52c0$1f0a0a0a@enfoldsystems.local> <46512B09.1080304@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE4B0@exchis.ccp.ad.local> <4652128B.3000301@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE6FB@exchis.ccp.ad.local> <465365DE.1090306@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEEACB@exchis.ccp.ad.local> <4E9372E6B2234D4F859320D896059A9508DE8B4046@exchis.ccp.ad.local> <465B53EF.6070108@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DE8B435C@exchis.ccp.ad.local> <465C9D09.9040401@v.loewis.de> Message-ID: <465CA75B.50901@ibp.de> Hi, >> One feature that is easily addable and will certainly make installing >> python on vista nicer, is to add authenticode signing to the install. I'm +1 on authenticode. > This I question very much. I experimented with authenticode before 2.4, > and found it an unacceptable experience. When the MSI file starts > running, installer needs to verify the signature, for which it needs > to compute a hash of the entire file. For the Python MSI, this takes > many seconds on a slower Pentium 4 machine. During that time, there > is no visual feedback, so users are uncertain whether they have > actually invoked the MSI file at all. > >> Currently the user is faced with a very nasty and off-putting message >> about an unidentified program requesting access to his computer. > > Certainly. However, telling them that they have to wait just so that > Windows finds out what they know already (that this is the MSI file > from the Python Software Foundation, or from Martin v. L?wis) is > even more nasty. Educated, adult developers with good internet connections may know that, but all users? What about software on a CD or a memory stick? Also, software sites/mirrors have been compromised in the past, and they are a sweet target. I haven't looked at authenticode, but I guess it's a cryptographical signature. That defaults to a good thing. That the verification takes time is unfortunate, but unavoidable. That the user interface sucks (no feedback) is a bug. You will have the say whether Python uses authenticode, but I'm not convinced by your arguments. - Lars From martin at v.loewis.de Wed May 30 07:10:20 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 30 May 2007 07:10:20 +0200 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <465CA75B.50901@ibp.de> References: <009f01c79b51$b1da52c0$1f0a0a0a@enfoldsystems.local> <46512B09.1080304@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE4B0@exchis.ccp.ad.local> <4652128B.3000301@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE6FB@exchis.ccp.ad.local> <465365DE.1090306@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEEACB@exchis.ccp.ad.local> <4E9372E6B2234D4F859320D896059A9508DE8B4046@exchis.ccp.ad.local> <465B53EF.6070108@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DE8B435C@exchis.ccp.ad.local> <465C9D09.9040401@v.loewis.de> <465CA75B.50901@ibp.de> Message-ID: <465D073C.6040304@v.loewis.de> >> Certainly. However, telling them that they have to wait just so that >> Windows finds out what they know already (that this is the MSI file >> from the Python Software Foundation, or from Martin v. L?wis) is >> even more nasty. > > Educated, adult developers with good internet connections may know that, > but all users? What about software on a CD or a memory stick? Also, I believe users *still* get a confirmation window, just the message changes from "we don't know who wrote this software" to "we know PSF wrote it - do you trust them?" So, "all users" aren't any better off with authenticode. > I haven't looked at authenticode, but I guess it's a cryptographical > signature. Correct. > That defaults to a good thing. That's a very common pitfall, and untrue. People are talked into believing that signed software is "more trustworthy" than unsigned software. This is absolutely not the case. The signed software may just as well contain malware. The only difference is that you can go after the author - provided you can get hold of him, and provided you can prove (in court) that it was actual that software that caused the damage. Depending on the malware, you may not even know that damage was made, e.g. if it was signed spyware. So code-signing can very realistically give a false sense of security. This is *not* a good thing. > You will have the say whether Python uses authenticode, but I'm not > convinced by your arguments. I think I'll have to produce a signed version of the 2.5.1 installer, so that people can see for themselves. Regards, Martin From lars at ibp.de Wed May 30 10:43:34 2007 From: lars at ibp.de (Lars Immisch) Date: Wed, 30 May 2007 10:43:34 +0200 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <465D073C.6040304@v.loewis.de> References: <009f01c79b51$b1da52c0$1f0a0a0a@enfoldsystems.local> <46512B09.1080304@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE4B0@exchis.ccp.ad.local> <4652128B.3000301@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE6FB@exchis.ccp.ad.local> <465365DE.1090306@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEEACB@exchis.ccp.ad.local> <4E9372E6B2234D4F859320D896059A9508DE8B4046@exchis.ccp.ad.local> <465B53EF.6070108@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DE8B435C@exchis.ccp.ad.local> <465C9D09.9040401@v.loewis.de> <465CA75B.50901@ibp.de> <465D073C.6040304@v.loewis.de> Message-ID: <465D3936.2020700@ibp.de> Dear Martin, >> Educated, adult developers with good internet connections may know that, >> but all users? What about software on a CD or a memory stick? > > Also, I believe users *still* get a confirmation window, just the > message changes from "we don't know who wrote this software" to > "we know PSF wrote it - do you trust them?" Ugh. Still better than a warning. > So, "all users" aren't any better off with authenticode. > >> I haven't looked at authenticode, but I guess it's a cryptographical >> signature. > > Correct. > >> That defaults to a good thing. > > That's a very common pitfall, and untrue. People are talked into > believing that signed software is "more trustworthy" than unsigned > software. This is absolutely not the case. The signed software may > just as well contain malware. The only difference is that you can > go after the author - provided you can get hold of him, and provided > you can prove (in court) that it was actual that software that > caused the damage. Depending on the malware, you may not even know > that damage was made, e.g. if it was signed spyware. Yes, I am aware of that. But the signature makes a man-in-the-middle attack harder. > So code-signing can very realistically give a false sense of > security. This is *not* a good thing. > >> You will have the say whether Python uses authenticode, but I'm not >> convinced by your arguments. > > I think I'll have to produce a signed version of the 2.5.1 installer, so > that people can see for themselves. That is a friendly idea. Thank you. - Lars From hjebbers at gmail.com Wed May 30 12:57:47 2007 From: hjebbers at gmail.com (henk-jan ebbers) Date: Wed, 30 May 2007 12:57:47 +0200 Subject: [Distutils] Windows installer, setuptools and distritools In-Reply-To: <465D3936.2020700@ibp.de> References: <009f01c79b51$b1da52c0$1f0a0a0a@enfoldsystems.local> <46512B09.1080304@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE4B0@exchis.ccp.ad.local> <4652128B.3000301@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE6FB@exchis.ccp.ad.local> <465365DE.1090306@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEEACB@exchis.ccp.ad.local> <4E9372E6B2234D4F859320D896059A9508DE8B4046@exchis.ccp.ad.local> <465B53EF.6070108@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DE8B435C@exchis.ccp.ad.local> <465C9D09.9040401@v.loewis.de> <465CA75B.50901@ibp.de> <465D073C.6040304@v.loewis.de> <465D3936.2020700@ibp.de> Message-ID: <465D58AB.7050504@gmail.com> I have 2 questions about using setuptools for a windows installer: 1. I want to make a executable-windows-installer with setuptools (bdist_wininst) . in setup.py I place a dependency: install_requires = ["TurboGears >= 1.0.1",], installation is not zip-safe: zip_safe=False, The target-machine (where I do the install) has easy_install already installed. Am I right that when I install this executable-windows-installer the dependency is not automatically installed? Or is there a way to do this with setuptools that I have overlooked? 2. in distritools it is possible to use a postinstallation script (*--install-script *option). (which I want to use shortcuts in menu's). This is not possible in setuptools? ( I just am not sure about this when reading the documentation. Eggs & setuptols are great; most user are used to windows installers, shortcuts in the menu etc.) best regards, Henk-Jan From tseaver at palladion.com Wed May 30 17:29:59 2007 From: tseaver at palladion.com (Tres Seaver) Date: Wed, 30 May 2007 11:29:59 -0400 Subject: [Distutils] I need to create a customized Fedora live cd with Python applications on it In-Reply-To: <20070529212504.665733A40A8@sparrow.telecommunity.com> References: <24685.207.188.248.157.1179867688.squirrel@www.cpcug.org> <20070522210151.AB6BE3A4061@sparrow.telecommunity.com> <38000.207.188.248.157.1179869031.squirrel@www.cpcug.org> <20070522220044.C82EA3A4061@sparrow.telecommunity.com> <31168.207.188.248.157.1180462247.squirrel@www.cpcug.org> <20070529194217.CEC093A409D@sparrow.telecommunity.com> <31648.207.188.248.157.1180470030.squirrel@www.cpcug.org> <20070529212504.665733A40A8@sparrow.telecommunity.com> Message-ID: <465D9877.9060504@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Phillip J. Eby wrote: > (By the way, you can always add those lines to the root user's > $HOME/.pydistutils.cfg instead of editing individual setup.cfg > files. Then, any packages installed (or RPMs built) by the root user > will include .pyo files.) Ugh. Please don't encourage anybody to *build* RPMs as root: that is a "worst pracice," encouraging broken packages, and risking nasty damage to the build system. Building as a non-rool user ensures that the .spec file uses a proper buildroot. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGXZh3+gerLs4ltQ4RAuX7AKC/HbPxxEmMzTV5s4BCjTwZ3/uaCwCgrJJc +PLq2Srr/b+ulCbUoyZb2pU= =Qu1/ -----END PGP SIGNATURE----- From tseaver at palladion.com Wed May 30 17:29:59 2007 From: tseaver at palladion.com (Tres Seaver) Date: Wed, 30 May 2007 11:29:59 -0400 Subject: [Distutils] I need to create a customized Fedora live cd with Python applications on it In-Reply-To: <20070529212504.665733A40A8@sparrow.telecommunity.com> References: <24685.207.188.248.157.1179867688.squirrel@www.cpcug.org> <20070522210151.AB6BE3A4061@sparrow.telecommunity.com> <38000.207.188.248.157.1179869031.squirrel@www.cpcug.org> <20070522220044.C82EA3A4061@sparrow.telecommunity.com> <31168.207.188.248.157.1180462247.squirrel@www.cpcug.org> <20070529194217.CEC093A409D@sparrow.telecommunity.com> <31648.207.188.248.157.1180470030.squirrel@www.cpcug.org> <20070529212504.665733A40A8@sparrow.telecommunity.com> Message-ID: <465D9877.9060504@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Phillip J. Eby wrote: > (By the way, you can always add those lines to the root user's > $HOME/.pydistutils.cfg instead of editing individual setup.cfg > files. Then, any packages installed (or RPMs built) by the root user > will include .pyo files.) Ugh. Please don't encourage anybody to *build* RPMs as root: that is a "worst pracice," encouraging broken packages, and risking nasty damage to the build system. Building as a non-rool user ensures that the .spec file uses a proper buildroot. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGXZh3+gerLs4ltQ4RAuX7AKC/HbPxxEmMzTV5s4BCjTwZ3/uaCwCgrJJc +PLq2Srr/b+ulCbUoyZb2pU= =Qu1/ -----END PGP SIGNATURE----- From pje at telecommunity.com Wed May 30 17:36:45 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 30 May 2007 11:36:45 -0400 Subject: [Distutils] Windows installer, setuptools and distritools In-Reply-To: <465D58AB.7050504@gmail.com> References: <009f01c79b51$b1da52c0$1f0a0a0a@enfoldsystems.local> <46512B09.1080304@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE4B0@exchis.ccp.ad.local> <4652128B.3000301@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE6FB@exchis.ccp.ad.local> <465365DE.1090306@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEEACB@exchis.ccp.ad.local> <4E9372E6B2234D4F859320D896059A9508DE8B4046@exchis.ccp.ad.local> <465B53EF.6070108@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DE8B435C@exchis.ccp.ad.local> <465C9D09.9040401@v.loewis.de> <465CA75B.50901@ibp.de> <465D073C.6040304@v.loewis.de> <465D3936.2020700@ibp.de> <465D58AB.7050504@gmail.com> Message-ID: <20070530154104.706623A40AC@sparrow.telecommunity.com> At 12:57 PM 5/30/2007 +0200, henk-jan ebbers wrote: >I have 2 questions about using setuptools for a windows installer: > >1. I want to make a executable-windows-installer with setuptools >(bdist_wininst) . > in setup.py I place a dependency: install_requires = >["TurboGears >= 1.0.1",], > installation is not zip-safe: zip_safe=False, > The target-machine (where I do the install) has easy_install already >installed. > Am I right that when I install this executable-windows-installer the >dependency is not automatically installed? That's correct. > Or is there a way to do this with setuptools that I have overlooked? > >2. in distritools it is possible to use a postinstallation script >(*--install-script *option). > (which I want to use shortcuts in menu's). > This is not possible in setuptools? It's possible with bdist_wininst, but if a user installs the .exe using easy_install, the postinstall script will be ignored. From pje at telecommunity.com Wed May 30 17:42:59 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 30 May 2007 11:42:59 -0400 Subject: [Distutils] I need to create a customized Fedora live cd with Python applications on it In-Reply-To: <465D9877.9060504@palladion.com> References: <24685.207.188.248.157.1179867688.squirrel@www.cpcug.org> <20070522210151.AB6BE3A4061@sparrow.telecommunity.com> <38000.207.188.248.157.1179869031.squirrel@www.cpcug.org> <20070522220044.C82EA3A4061@sparrow.telecommunity.com> <31168.207.188.248.157.1180462247.squirrel@www.cpcug.org> <20070529194217.CEC093A409D@sparrow.telecommunity.com> <31648.207.188.248.157.1180470030.squirrel@www.cpcug.org> <20070529212504.665733A40A8@sparrow.telecommunity.com> <465D9877.9060504@palladion.com> Message-ID: <20070530154105.4A7173A40AE@sparrow.telecommunity.com> At 11:29 AM 5/30/2007 -0400, Tres Seaver wrote: >-----BEGIN PGP SIGNED MESSAGE----- >Hash: SHA1 > >Phillip J. Eby wrote: > > > (By the way, you can always add those lines to the root user's > > $HOME/.pydistutils.cfg instead of editing individual setup.cfg > > files. Then, any packages installed (or RPMs built) by the root user > > will include .pyo files.) > >Ugh. Please don't encourage anybody to *build* RPMs as root: that is >a "worst pracice," encouraging broken packages, and risking nasty damage >to the build system. Building as a non-rool user ensures that the .spec >file uses a proper buildroot. I'm was just noting that if the bytecode compilation is a post-install step, the config would have to be in the root user's $HOME. But in truth, that's not actually necessary, now that I'm thinking about it. The key is whether the .pyo files get listed in the installed files list, and that's determined by the builder, not the installer, IIUC. From hjebbers at gmail.com Wed May 30 18:03:29 2007 From: hjebbers at gmail.com (henk-jan ebbers) Date: Wed, 30 May 2007 18:03:29 +0200 Subject: [Distutils] Windows installer, setuptools and distritools In-Reply-To: <20070530154104.706623A40AC@sparrow.telecommunity.com> References: <009f01c79b51$b1da52c0$1f0a0a0a@enfoldsystems.local> <46512B09.1080304@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE4B0@exchis.ccp.ad.local> <4652128B.3000301@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE6FB@exchis.ccp.ad.local> <465365DE.1090306@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEEACB@exchis.ccp.ad.local> <4E9372E6B2234D4F859320D896059A9508DE8B4046@exchis.ccp.ad.local> <465B53EF.6070108@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DE8B435C@exchis.ccp.ad.local> <465C9D09.9040401@v.loewis.de> <465CA75B.50901@ibp.de> <465D073C.6040304@v.loewis.de> <465D3936.2020700@ibp.de> <465D58AB.7050504@gmail.com> <20070530154104.706623A40AC@sparrow.telecommunity.com> Message-ID: <465DA051.1010802@gmail.com> So: either a user uses the executable-windows-installer (with postinstallation script, but dependencies are not resolved), or a user uses easy_install (without postinstallation script, but dependencies are resolved) Is it possible to place something like 'easy_install ' in the postinstallation script? best regards, Henk-Jan Phillip J. Eby wrote: > At 12:57 PM 5/30/2007 +0200, henk-jan ebbers wrote: >> I have 2 questions about using setuptools for a windows installer: >> >> 1. I want to make a executable-windows-installer with setuptools >> (bdist_wininst) . >> in setup.py I place a dependency: install_requires = >> ["TurboGears >= 1.0.1",], >> installation is not zip-safe: zip_safe=False, >> The target-machine (where I do the install) has easy_install already >> installed. >> Am I right that when I install this executable-windows-installer the >> dependency is not automatically installed? > > That's correct. > > >> Or is there a way to do this with setuptools that I have overlooked? >> >> 2. in distritools it is possible to use a postinstallation script >> (*--install-script *option). >> (which I want to use shortcuts in menu's). >> This is not possible in setuptools? > > It's possible with bdist_wininst, but if a user installs the .exe > using easy_install, the postinstall script will be ignored. > > From pje at telecommunity.com Wed May 30 18:22:02 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 30 May 2007 12:22:02 -0400 Subject: [Distutils] Windows installer, setuptools and distritools In-Reply-To: <465DA051.1010802@gmail.com> References: <009f01c79b51$b1da52c0$1f0a0a0a@enfoldsystems.local> <46512B09.1080304@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE4B0@exchis.ccp.ad.local> <4652128B.3000301@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE6FB@exchis.ccp.ad.local> <465365DE.1090306@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEEACB@exchis.ccp.ad.local> <4E9372E6B2234D4F859320D896059A9508DE8B4046@exchis.ccp.ad.local> <465B53EF.6070108@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DE8B435C@exchis.ccp.ad.local> <465C9D09.9040401@v.loewis.de> <465CA75B.50901@ibp.de> <465D073C.6040304@v.loewis.de> <465D3936.2020700@ibp.de> <465D58AB.7050504@gmail.com> <20070530154104.706623A40AC@sparrow.telecommunity.com> <465DA051.1010802@gmail.com> Message-ID: <20070530162011.C62FD3A40A3@sparrow.telecommunity.com> At 06:03 PM 5/30/2007 +0200, henk-jan ebbers wrote: >So: >either a user uses the executable-windows-installer (with >postinstallation script, but dependencies are not resolved), >or a user uses easy_install (without postinstallation script, >but dependencies are resolved) > >Is it possible to place something like 'easy_install ' >in the postinstallation script? Sure, although of course those dependencies won't have *their* post-install scripts run. If you do this, you probably want to have the postinstall script register the files installed. easy_install's --record option lets you write the installed files and directories to a text file, so you could read that and record the results. That way, if the .exe is uninstalled, the other files will go along. The downside to this is that uninstalling the .exe will uninstall dependencies that might actually be depended on by other packages that are installed later -- IOW, a total mess. So, I'm not sure it's a good idea. No, scratch that, I'm pretty sure it's NOT a good idea. It might be better to use bdist_msi instead, if that has a way of handling dependencies. (I'm not sure if it does.) From hjebbers at gmail.com Wed May 30 21:41:18 2007 From: hjebbers at gmail.com (henk-jan ebbers) Date: Wed, 30 May 2007 21:41:18 +0200 Subject: [Distutils] Windows installer, setuptools and distritools In-Reply-To: <20070530162011.C62FD3A40A3@sparrow.telecommunity.com> References: <009f01c79b51$b1da52c0$1f0a0a0a@enfoldsystems.local> <46512B09.1080304@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE4B0@exchis.ccp.ad.local> <4652128B.3000301@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE6FB@exchis.ccp.ad.local> <465365DE.1090306@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEEACB@exchis.ccp.ad.local> <4E9372E6B2234D4F859320D896059A9508DE8B4046@exchis.ccp.ad.local> <465B53EF.6070108@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DE8B435C@exchis.ccp.ad.local> <465C9D09.9040401@v.loewis.de> <465CA75B.50901@ibp.de> <465D073C.6040304@v.loewis.de> <465D3936.2020700@ibp.de> <465D58AB.7050504@gmail.com> <20070530154104.706623A40AC@sparrow.telecommunity.com> <465DA051.1010802@gmail.com> <20070530162011.C62FD3A40A3@sparrow.telecommunity.com> Message-ID: <465DD35E.2020206@gmail.com> An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20070530/693d44f5/attachment-0001.htm From martin at v.loewis.de Wed May 30 22:24:25 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 30 May 2007 22:24:25 +0200 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <465D3936.2020700@ibp.de> References: <009f01c79b51$b1da52c0$1f0a0a0a@enfoldsystems.local> <46512B09.1080304@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE4B0@exchis.ccp.ad.local> <4652128B.3000301@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE6FB@exchis.ccp.ad.local> <465365DE.1090306@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEEACB@exchis.ccp.ad.local> <4E9372E6B2234D4F859320D896059A9508DE8B4046@exchis.ccp.ad.local> <465B53EF.6070108@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DE8B435C@exchis.ccp.ad.local> <465C9D09.9040401@v.loewis.de> <465CA75B.50901@ibp.de> <465D073C.6040304@v.loewis.de> <465D3936.2020700@ibp.de> Message-ID: <465DDD79.2030404@v.loewis.de> >>> Educated, adult developers with good internet connections may know that, >>> but all users? What about software on a CD or a memory stick? >> >> Also, I believe users *still* get a confirmation window, just the >> message changes from "we don't know who wrote this software" to >> "we know PSF wrote it - do you trust them?" > > Ugh. Still better than a warning. It's still a warning. >> That's a very common pitfall, and untrue. People are talked into >> believing that signed software is "more trustworthy" than unsigned >> software. This is absolutely not the case. The signed software may >> just as well contain malware. The only difference is that you can >> go after the author - provided you can get hold of him, and provided >> you can prove (in court) that it was actual that software that >> caused the damage. Depending on the malware, you may not even know >> that damage was made, e.g. if it was signed spyware. > > Yes, I am aware of that. But the signature makes a man-in-the-middle > attack harder. Sure. However, I think that this protection against an unlikely scenario cannot outweigh the main problem of code signing: that people get a false sense of security. And trust (sic) me: they do. Regards, Martin From kristjan at ccpgames.com Wed May 30 23:41:10 2007 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=) Date: Wed, 30 May 2007 21:41:10 +0000 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <465DDD79.2030404@v.loewis.de> References: <009f01c79b51$b1da52c0$1f0a0a0a@enfoldsystems.local> <46512B09.1080304@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE4B0@exchis.ccp.ad.local> <4652128B.3000301@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE6FB@exchis.ccp.ad.local> <465365DE.1090306@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEEACB@exchis.ccp.ad.local> <4E9372E6B2234D4F859320D896059A9508DE8B4046@exchis.ccp.ad.local> <465B53EF.6070108@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DE8B435C@exchis.ccp.ad.local> <465C9D09.9040401@v.loewis.de> <465CA75B.50901@ibp.de> <465D073C.6040304@v.loewis.de> <465D3936.2020700@ibp.de> <465DDD79.2030404@v.loewis.de> Message-ID: <4E9372E6B2234D4F859320D896059A9508DE8B4779@exchis.ccp.ad.local> > -----Original Message----- > From: "Martin v. L?wis" [mailto:martin at v.loewis.de] > > > > Yes, I am aware of that. But the signature makes a man-in-the-middle > > attack harder. > > Sure. However, I think that this protection against an unlikely > scenario > cannot outweigh the main problem of code signing: that people get a > false sense of security. And trust (sic) me: they do. > Us trying to lecture MS on software security is pointless. The current install will give users a sense of insecurity, false or otherwise. Kristjan From martin at v.loewis.de Thu May 31 00:38:17 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 31 May 2007 00:38:17 +0200 Subject: [Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <4E9372E6B2234D4F859320D896059A9508DE8B4779@exchis.ccp.ad.local> References: <009f01c79b51$b1da52c0$1f0a0a0a@enfoldsystems.local> <46512B09.1080304@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE4B0@exchis.ccp.ad.local> <4652128B.3000301@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEE6FB@exchis.ccp.ad.local> <465365DE.1090306@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DCBEEACB@exchis.ccp.ad.local> <4E9372E6B2234D4F859320D896059A9508DE8B4046@exchis.ccp.ad.local> <465B53EF.6070108@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DE8B435C@exchis.ccp.ad.local> <465C9D09.9040401@v.loewis.de> <465CA75B.50901@ibp.de> <465D073C.6040304@v.loewis.de> <465D3936.2020700@ibp.de> <465DDD79.2030404@v.loewis.de> <4E9372E6B2234D4F859320D896059A9508DE8B4779@exchis.ccp.ad.local> Message-ID: <465DFCD9.4020906@v.loewis.de> > Us trying to lecture MS on software security is pointless. I didn't try to lecture MS. I explained that to Lars Immisch, who essentially said that authenticode is a good thing because it is based on certificates. Regards, Martin From pje at telecommunity.com Thu May 31 19:56:32 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 31 May 2007 13:56:32 -0400 Subject: [Distutils] setuptools 0.6c6 released Message-ID: <20070531175438.77A2B3A40AC@sparrow.telecommunity.com> I've uploaded a new release version of setuptools, 0.6c6, which fixes a number of outstanding problems. Please note that the following issues are *not* yet fixed: * "bdist_wininst upload" on Python 2.5 sometimes tries to upload the .exe twice * platform-specific eggs don't take precedence over non-platform-specific ones * error checking of regular+namespace packages isn't sufficient (you can still specify silly things and get silly results) * data files specifed only via package_data may not be included in a generated sdist This is probably not a complete list, either. I've got a much longer TODO list, but most of the rest are more like "quirks" than bugs.