From vivian at vdesmedt.com Sat Apr 2 08:37:33 2005 From: vivian at vdesmedt.com (Vivian De Smedt) Date: Sat Apr 2 08:37:35 2005 Subject: [Distutils] small improvement of the script functionality under win32 Message-ID: <424E3DAD.10306@vdesmedt.com> Here is a small summary of the remark made arround the she-bang modification of the windows platform. 1. Trent let us know that the she-bang modification works only for the win nt sons and that in general it is better to use the .bat extension instead of the .cmd one 2. I realize that we shouldn't change the extension of a script if it exist since the extention could be usefull to determine which version of python to run - pythonw for the .pyw extention - python for the .py extention According to these remarks and in the purpose to be as conservative as possible I make this another proposition: The change in the she bang will happend only if - the win32 platform is detected (os.name == "nt") - the platform is win NT son (os.environ["OS"] == "Windows_NT") - the original script had no extension (os.path.splitext(outfile)[1] == "") Furthermore I add the %* in the new nt she-bang to let the underlying script know about command line argument. such that the unix she-bang: #!python foo bar will become: @C:\Python24\Python.exe -x "%%~f0" foo bar %* & exit /b I think that in these conditions the change could only improve the existing situation at least it improve for some them (Zope, Chetaah, PyCrust, IPython) Tell me what you think about the proposed change. Vivian. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/distutils-sig/attachments/20050402/40773b70/attachment.html From calvin at users.sourceforge.net Sat Apr 2 09:58:57 2005 From: calvin at users.sourceforge.net (Bastian Kleineidam) Date: Sat Apr 2 09:59:00 2005 Subject: [Distutils] small improvement of the script functionality under win32 In-Reply-To: <424E3DAD.10306@vdesmedt.com> References: <424E3DAD.10306@vdesmedt.com> Message-ID: <20050402075857.GA14741@treasure> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, On Sat, Apr 02, 2005 at 08:37:33AM +0200, Vivian De Smedt wrote: > The change in the she bang will happend only if > - the win32 platform is detected (os.name == "nt") > - the platform is win NT son (os.environ["OS"] == "Windows_NT") I suggest using the platform module: import platform win_nt_releases = ['NT', 'XP', '2000', '2003Server'] if platform.system() == 'Windows' and platform.release() in win_nt_releases: # do it Regards, Bastian - -- ,''`. Bastian Kleineidam : :' : GnuPG Schl?ssel `. `' gpg --keyserver wwwkeys.pgp.net --recv-keys 32EC6F3E `- -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.0 (GNU/Linux) iD8DBQFCTlDBeBwlBDLsbz4RAhAhAJ9+zAtHAQOanZssYyGXUlzOD17YCACfT9oS hC5qSjmJtv9aPIuY3zSphRo= =3XD4 -----END PGP SIGNATURE----- From pje at telecommunity.com Sat Apr 2 10:06:30 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat Apr 2 10:02:48 2005 Subject: [Distutils] small improvement of the script functionality under win32 In-Reply-To: <424E3DAD.10306@vdesmedt.com> Message-ID: <5.1.1.6.0.20050402030427.02a01950@mail.telecommunity.com> At 08:37 AM 4/2/05 +0200, Vivian De Smedt wrote: >The change in the she bang will happend only if > - the win32 platform is detected (os.name == "nt") > - the platform is win NT son (os.environ["OS"] == "Windows_NT") You really need to check that COMSPEC points to cmd.exe, since as someone pointed out you can actually have an NT machine using command.com as its COMSPEC, which means it won't be able to run this either. And whatever environment variable, you need to handle the case where it doesn't exist. My Win 98 machine has no 'OS' variable, for example. From p.f.moore at gmail.com Sat Apr 2 11:45:40 2005 From: p.f.moore at gmail.com (Paul Moore) Date: Sat Apr 2 11:45:43 2005 Subject: [Distutils] small improvement of the script functionality under win32 In-Reply-To: <5.1.1.6.0.20050402030427.02a01950@mail.telecommunity.com> References: <424E3DAD.10306@vdesmedt.com> <5.1.1.6.0.20050402030427.02a01950@mail.telecommunity.com> Message-ID: <79990c6b050402014538305a7@mail.gmail.com> On Apr 2, 2005 9:06 AM, Phillip J. Eby wrote: > At 08:37 AM 4/2/05 +0200, Vivian De Smedt wrote: > > >The change in the she bang will happend only if > > - the win32 platform is detected (os.name == "nt") > > - the platform is win NT son (os.environ["OS"] == "Windows_NT") > > You really need to check that COMSPEC points to cmd.exe, since as someone > pointed out you can actually have an NT machine using command.com as its > COMSPEC, which means it won't be able to run this either. And just to be annoying, my COMSPEC (sometimes) points to 4NT.EXE, and I can use this feature. I support the idea, but it may not be practical. To be honest, shipping the files with a .py extension (which is directly associated with Python) in the first place is probably a better answer. Paul. From pje at telecommunity.com Sat Apr 2 21:52:58 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat Apr 2 21:49:22 2005 Subject: [Distutils] small improvement of the script functionality under win32 In-Reply-To: <79990c6b050402014538305a7@mail.gmail.com> References: <5.1.1.6.0.20050402030427.02a01950@mail.telecommunity.com> <424E3DAD.10306@vdesmedt.com> <5.1.1.6.0.20050402030427.02a01950@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20050402144559.03b50700@mail.telecommunity.com> At 10:45 AM 4/2/05 +0100, Paul Moore wrote: >On Apr 2, 2005 9:06 AM, Phillip J. Eby wrote: > > At 08:37 AM 4/2/05 +0200, Vivian De Smedt wrote: > > > > >The change in the she bang will happend only if > > > - the win32 platform is detected (os.name == "nt") > > > - the platform is win NT son (os.environ["OS"] == "Windows_NT") > > > > You really need to check that COMSPEC points to cmd.exe, since as someone > > pointed out you can actually have an NT machine using command.com as its > > COMSPEC, which means it won't be able to run this either. > >And just to be annoying, my COMSPEC (sometimes) points to 4NT.EXE, and >I can use this feature. > >I support the idea, but it may not be practical. To be honest, >shipping the files with a .py extension (which is directly associated >with Python) in the first place is probably a better answer. Actually, with the availability of -m, it probably makes more sense to add the ability to specify "script modules", mapping from a desired command name to a module to be run as __main__. Then, you can generate a platform-specific stub to run "python -m modulename". On Windows that could be a .bat, and on Unixish systems there could be no extension. This wouldn't help existing distributions with scripts, or anything for Python <2.4, but it seems like the "right" way to fix the problem in the long term. From vivian at vdesmedt.com Sat Apr 2 22:36:12 2005 From: vivian at vdesmedt.com (Vivian De Smedt) Date: Sat Apr 2 22:36:14 2005 Subject: [Distutils] small improvement of the script functionality under win32 In-Reply-To: <5.1.1.6.0.20050402144559.03b50700@mail.telecommunity.com> References: <5.1.1.6.0.20050402030427.02a01950@mail.telecommunity.com> <424E3DAD.10306@vdesmedt.com> <5.1.1.6.0.20050402030427.02a01950@mail.telecommunity.com> <5.1.1.6.0.20050402144559.03b50700@mail.telecommunity.com> Message-ID: <424F023C.8040901@vdesmedt.com> I'm sorry to be so insistent but it seems to me that my second proposition wasn't well understood. My goal wasn't to find the perfect solution but just to improve the existing solution in particular for the scripts without extensions. 1) Here is a small "demonstration" of my point a demonstration that just ask to be contradict :-) Indeed when a script have no extension it have no chance to be understood by any of the version of the win32 platform so it difficult to imagine any drawback to a modification of its she-bang. Furthermore on all version of NT it provide a working alternative with the consequence that the provided patch have benefits and no drawbacks. I everybody agree with this lets talk of the other scripts we could think to modify (the one with extension) or the other sub win32 platform we could try to find a solution for. 2) The script with extension (.py, .pyw) I think that the scripts with extension are pretty well managed on the win32 platform. The only drawback of the actual solution is that the redirection of the input isn't working and for most scripts it do not seems to me a big problem. Furthermore the extension allows the system to select which version of python to run, pythonw.exe for .pyw or python.exe for .py for some of us or, python.exe for both .py and .pyw for the one that like to see the logs of the .pyw scripts. At first glance the little advantage combined with a big drawback (but this is quit subjective) let me think that we shouldn't try to modify the extension of the scripts that have already one. 3) The other sub platforms (Dos, Win95, Win98) I do have any satisfactory solution maybe the proposition of Phillip J. Eby contains idea but not having access to any of these platform I'm in a bad position to be able to propose any solutions. Maybe: - adding a .py extension o - adding a .bat extension and a variation of the she-bang we propose for the nt sub platforms are two improvements interesting improvement we could consider. The first could also apply to the nt sub platform I personally like more my original one since it let the developer the possibility to provide scripts that support redirection of the input. Again sorry to be so insistant and so long being probably so unclear in my previous posts. Vivian. From pje at telecommunity.com Sat Apr 2 23:05:21 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat Apr 2 23:01:44 2005 Subject: [Distutils] small improvement of the script functionality under win32 In-Reply-To: <424F023C.8040901@vdesmedt.com> References: <5.1.1.6.0.20050402144559.03b50700@mail.telecommunity.com> <5.1.1.6.0.20050402030427.02a01950@mail.telecommunity.com> <424E3DAD.10306@vdesmedt.com> <5.1.1.6.0.20050402030427.02a01950@mail.telecommunity.com> <5.1.1.6.0.20050402144559.03b50700@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20050402160141.02f22b70@mail.telecommunity.com> At 10:36 PM 4/2/05 +0200, Vivian De Smedt wrote: >3) The other sub platforms (Dos, Win95, Win98) > >I do have any satisfactory solution maybe the proposition of Phillip J. >Eby contains idea but not having access to any of these platform I'm in a >bad position to be able to propose any solutions. These could be resolved by creating a *second* file with a .bat extension that invokes python on the file without the extension, and passed the arguments, e.g.: @echo off shift C:\Python24\Python C:\Python24\Scripts\scriptnamehere %0 %1 %2 %3 %4 %5 %6 %7 %8 %9 This should work on DOS and Windows 98; perhaps someone else can double check that it works on NT-based systems. If so, then this is a viable approach for all win32 platforms. From vivian at vdesmedt.com Sun Apr 3 08:10:04 2005 From: vivian at vdesmedt.com (Vivian De Smedt) Date: Sun Apr 3 08:10:06 2005 Subject: [Distutils] small improvement of the script functionality under win32 In-Reply-To: <5.1.1.6.0.20050402160141.02f22b70@mail.telecommunity.com> References: <5.1.1.6.0.20050402144559.03b50700@mail.telecommunity.com> <5.1.1.6.0.20050402030427.02a01950@mail.telecommunity.com> <424E3DAD.10306@vdesmedt.com> <5.1.1.6.0.20050402030427.02a01950@mail.telecommunity.com> <5.1.1.6.0.20050402144559.03b50700@mail.telecommunity.com> <5.1.1.6.0.20050402160141.02f22b70@mail.telecommunity.com> Message-ID: <424F88BC.7010009@vdesmedt.com> 1) Comment on the Phillip J. Eby proposition: It works well on the nt platform. - script execution - arguments passing (if less that10 :-) I suppose that the win95, win98 platform do not support the %* feature) - redirection of the input - redirection of the output - return value (%errorlevel%) I slightly prefere the following version: @shift @python script.py %0 %1 %2 %3 %4 %5 %6 %7 %8 %9 because it do not change the echo flag. 2) Advantage / Drawbacks: The advantage that I'am able to see are: - cross sub platform compatibility - close to what have been done by some package manager The drawback of this solution that I can foreseen are: - the scripts can accept only 10 arguments - the scripts could not easily be relocated Because I have too take cross platform into consideration and because I don't think relocating these script is really an issue I think the solution of Phillip J. Eby is better. 3) Small side remark. If we go for a solution that output an new file or a one that wasn't expected by the developper (script -> script.bat or script -> script and script.bat) we should cross check that the developper haven't already plan to ouput script.bat. I says that because some team did choose the solution of Phillip J. Eby with variations using sometime python and sometime pythonw according to the goal of their scripts and we should let them the flexibility to do that. Again this is just a side remark because whatever solution will be choosed we should probably take care of that. Vivian. From public-dstml at giss.ath.cx Sun Apr 3 09:47:13 2005 From: public-dstml at giss.ath.cx (AGiss) Date: Sun Apr 3 09:47:22 2005 Subject: [Distutils] Mingw Python 2.4 Message-ID: <20050403074713.GA14413@gissehel.mine.nu> Hello, I tried to compile some extensions under Windows/Python 2.4 using distutils and mingw as compiler (not MS-VC++). ----- (You can just jump to conclusion at the bottom if you like, but be aware that I don't really master the subject, so I may have missed something. If you thing my conclusions doesn't make sense, please read what happens to me.) ----- (Official Python2.4, the one that use msvcr71.dll) I got some strange results like "the softwares I wrote using this extension crashed randomly". I first thought that extension (that I didn't wrote) was crappy. When the second extension crashed too, I tried to understand a bit more. The crash is more or less random, (but when it doesn't crash, everything works as expected even the code in the extension) and it crash with an OS error 0xc0000008 (it look like it's an internal windows error which mean there is something really nasty with the memory. So the .pyd was linked to msvcrt.dll and to msvcr71.dll. As it looks fishy, I tried to reinstall python 2.3 for windows that doesn't use msvcr71.dll but msvcrt.dll). I wasn't able to reproduce the crash. Then I realize the extension provided a way to build the same .pyd (but I wasn't able to generate an installer for the extension, that's why I didn't use it). So I look at the .pyd generated for python 2.4 with the makefile, and... it was only linked to msvcrt.dll. So I trie to use it instead and... it works !!! I tried to recompile everything from scratch using distutils because it may just be because a "bad neutrino who hit my motherboard at the worst moment", but the bug was still here. So I tried to figure out what was the difference between the command lines used by the makefile and distutils. And... the obvious difference was that distutils used a -lmsvcr71. Gooood I looked at distutils code source, and quickly found the line that makes mingw use msvcr71. IV just changed it. I rebuild all, and... It just worked well. (and of course, the .pyd was just linked to msvcrt and not msvcr71 anymore) -------------- Sooooooooooooooooo here are my conclusions : Something isn't right when compiling with mingw using msvcr71. (command line option ? problem with mingw ? problem with msvcr71 ? mingw just need some more configuration to works with msvcr71 ? I don't really know) It *look* like compilling extension using msvcrt instad of msvcr71, even if python use msvcr71 is safer that letting mingw using msvcr71. So I propose to remove the lines from 330 to 332 on the file cygwinccompiler.py (revision 1.29, the actual one, and the one used with python 2.4) The code is specific for Windows/Mingw/(python compiled with msvc 7.1) so there shouldn't be any side effect. Perahps it's wise to do the same for the line that add "msvcr70" (but I don't know). Perhaps it's wise to do the same on lines 130-135 (The same code, but for cygwin's internal gcc, and not mingw's gcc), but I don't know if cygwin suffer for the same problems. From mdehoon at ims.u-tokyo.ac.jp Sun Apr 3 11:08:34 2005 From: mdehoon at ims.u-tokyo.ac.jp (Michiel Jan Laurens de Hoon) Date: Sun Apr 3 11:03:48 2005 Subject: [Distutils] Mingw Python 2.4 In-Reply-To: <20050403074713.GA14413@gissehel.mine.nu> References: <20050403074713.GA14413@gissehel.mine.nu> Message-ID: <424FB292.5080008@ims.u-tokyo.ac.jp> I just tried to compile a bunch of C extensions (coming from the biopython project) with mingw as compiler. Most of them link to msvcr71.dll and not to msvcrt.dll, except for one, which links to both. By uncommenting various parts of the code, I discovered that the culprit is a call to strdup. When such a call is made, the extension module will link to both msvcr71.dll and msvcrt.dll. Now, strdup is not part of ANSI-C, so for portability it is better to avoid this function altogether. I don't know if your extension module links to msvcrt.dll for the same reason, but you may be able to find out which part of the code is causing the .pyd to be linked to msvcrt. It may turn out to be strdup or another non-standard function. --Michiel. AGiss wrote: > Hello, > > I tried to compile some extensions under Windows/Python > 2.4 using distutils and mingw as compiler (not MS-VC++). > > ----- > (You can just jump to conclusion at the bottom if you like, > but be aware that I don't really master the subject, so > I may have missed something. If you thing my conclusions > doesn't make sense, please read what happens to me.) > ----- > > (Official Python2.4, the one that use msvcr71.dll) > > I got some strange results like "the softwares I wrote using this > extension crashed randomly". I first thought that extension > (that I didn't wrote) was crappy. > > When the second extension crashed too, I tried to understand > a bit more. > > The crash is more or less random, (but when it doesn't crash, > everything works as expected even the code in the extension) > and it crash with an OS error 0xc0000008 (it look like it's > an internal windows error which mean there is something really > nasty with the memory. > > So the .pyd was linked to msvcrt.dll and to msvcr71.dll. > As it looks fishy, I tried to reinstall python 2.3 for > windows that doesn't use msvcr71.dll but msvcrt.dll). > > I wasn't able to reproduce the crash. > > Then I realize the extension provided a way to build the same > .pyd (but I wasn't able to generate an installer for the > extension, that's why I didn't use it). > > So I look at the .pyd generated for python 2.4 with the > makefile, and... it was only linked to msvcrt.dll. > So I trie to use it instead and... it works !!! > > I tried to recompile everything from scratch using distutils > because it may just be because a "bad neutrino who hit my > motherboard at the worst moment", but the bug was still here. > > So I tried to figure out what was the difference between the > command lines used by the makefile and distutils. And... the > obvious difference was that distutils used a -lmsvcr71. > > Gooood > > I looked at distutils code source, and quickly found the line > that makes mingw use msvcr71. IV just changed it. > > I rebuild all, and... It just worked well. (and of course, the > .pyd was just linked to msvcrt and not msvcr71 anymore) > > -------------- > > Sooooooooooooooooo here are my conclusions : > > Something isn't right when compiling with mingw using msvcr71. > (command line option ? problem with mingw ? problem with > msvcr71 ? mingw just need some more configuration to works > with msvcr71 ? I don't really know) > > It *look* like compilling extension using msvcrt instad of > msvcr71, even if python use msvcr71 is safer that letting > mingw using msvcr71. > > So I propose to remove the lines from 330 to 332 on the file > cygwinccompiler.py (revision 1.29, the actual one, and the > one used with python 2.4) > > The code is specific for Windows/Mingw/(python compiled with msvc 7.1) > so there shouldn't be any side effect. > > Perahps it's wise to do the same for the line that add "msvcr70" > (but I don't know). > > Perhaps it's wise to do the same on lines 130-135 (The same code, > but for cygwin's internal gcc, and not mingw's gcc), but I don't > know if cygwin suffer for the same problems. > > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG@python.org > http://mail.python.org/mailman/listinfo/distutils-sig > > -- Michiel de Hoon, Assistant Professor University of Tokyo, Institute of Medical Science Human Genome Center 4-6-1 Shirokane-dai, Minato-ku Tokyo 108-8639 Japan http://bonsai.ims.u-tokyo.ac.jp/~mdehoon From public-dstml at giss.ath.cx Sun Apr 3 13:32:34 2005 From: public-dstml at giss.ath.cx (AGiss) Date: Sun Apr 3 13:32:37 2005 Subject: [Distutils] Mingw Python 2.4 In-Reply-To: <424FB292.5080008@ims.u-tokyo.ac.jp> References: <20050403074713.GA14413@gissehel.mine.nu> <424FB292.5080008@ims.u-tokyo.ac.jp> Message-ID: <20050403113234.GA15618@gissehel.mine.nu> On Sun, Apr 03, 2005 at 06:08:34PM +0900, Michiel Jan Laurens de Hoon wrote: > I just tried to compile a bunch of C extensions (coming from the biopython > project) with mingw as compiler. Most of them link to msvcr71.dll and not > to msvcrt.dll, except for one, which links to both. By uncommenting various > parts of the code, I discovered that the culprit is a call to strdup. When > such a call is made, the extension module will link to both msvcr71.dll and > msvcrt.dll. Now, strdup is not part of ANSI-C, so for portability it is > better to avoid this function altogether. I don't know if your extension > module links to msvcrt.dll for the same reason, but you may be able to find > out which part of the code is causing the .pyd to be linked to msvcrt. It > may turn out to be strdup or another non-standard function. > > --Michiel. The extension is "clearsilver" (http://www.clearsilver.net/). It's a "third-party" extension, so "not using" strdup is not a "clean" solution. (while in this case, rewritting it is quite simple, so it didn't took me much time to do it). So the .pyd generated with distutils linked to msvcrt.dll and to msvcr71.dll calls linked to msvcrt.dll : _close _fdopen _mkdir _mktemp _open _putenv _read _rmdir _sleep _stat _strdup _tzset _unlink _write _findclose _findfirst _findnext _fullpath calls linked to msvcr71.dll : __dllonexit __mb_cur_max _errno _iob _isctype _pctype _stricmp _strnicmp _vsnprintf abort atoi bsearch calloc fclose fflush fgets fopen fprintf fread free fseek fwrite getenv gmtime localtime malloc memchr memcpy memmove memset mktime qsort rand realloc rename sprintf srand sscanf strcat strchr strcmp strcpy strerror strftime strlen strncat strncmp strncpy strpbrk strrchr strspn strstr strtol time tolower toupper vfprintf vprintf vsprintf while all calls in the first section ARE exported in msvcr71.dll too (so that's not a problem of "msvcr71.dll" doesn't have them) (I suppose that msvcr71.dll export ALL the items mscvrt.dll plus some more). And so yes, _stdup is in the list. I tried to remove the strdup calls (just in case strdup would be the *only* "trigger" to that weirdness). So I replace all calls to strdup to strxdup, and I wrote a quick strxdup using just "strlen", "malloc" and "strncpy". The result was ... _strdup wasn't in the list of dependences from neo_cgi.pyd to msvcrt.dll. But all others were still remaining. And so in the list, if unlink, sleep or fdopen aren't ANSI-C, they look like POSIX to me. So I feel like using them. Am I wrong ? So I don't know what is really happening (look like MingW32 has problems handling msvcr71.dll, but I'm far from being competent to say that), but I feel like distutils should do something to not generate .pyd that would crash python scripts. (perhaps generate .pyd with mscvr71, then look if .pyd is linked to several run-time libs, and if yes, "downgrade" to mscvrt ?) --AGiss From pje at telecommunity.com Sun Apr 3 16:23:49 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun Apr 3 16:20:26 2005 Subject: [Distutils] small improvement of the script functionality under win32 In-Reply-To: <424F89CE.9000206@vdesmedt.com> References: <5.1.1.6.0.20050402160141.02f22b70@mail.telecommunity.com> <5.1.1.6.0.20050402144559.03b50700@mail.telecommunity.com> <5.1.1.6.0.20050402030427.02a01950@mail.telecommunity.com> <424E3DAD.10306@vdesmedt.com> <5.1.1.6.0.20050402030427.02a01950@mail.telecommunity.com> <5.1.1.6.0.20050402144559.03b50700@mail.telecommunity.com> <5.1.1.6.0.20050402160141.02f22b70@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20050403102214.026609f0@mail.telecommunity.com> At 08:14 AM 4/3/05 +0200, Vivian De Smedt wrote: >Could you confirm me that the %* isn't working on win95/win98 ? test.bat: @echo off echo %* Running 'test test' Outputs an '*', not any arguments. So no, it doesn't work. From trentm at ActiveState.com Mon Apr 4 20:26:21 2005 From: trentm at ActiveState.com (Trent Mick) Date: Mon Apr 4 20:31:00 2005 Subject: [Distutils] small improvement of the script functionality under win32 In-Reply-To: <5.1.1.6.0.20050403102214.026609f0@mail.telecommunity.com> References: <5.1.1.6.0.20050402160141.02f22b70@mail.telecommunity.com> <5.1.1.6.0.20050402144559.03b50700@mail.telecommunity.com> <5.1.1.6.0.20050402030427.02a01950@mail.telecommunity.com> <424E3DAD.10306@vdesmedt.com> <5.1.1.6.0.20050402030427.02a01950@mail.telecommunity.com> <5.1.1.6.0.20050402144559.03b50700@mail.telecommunity.com> <5.1.1.6.0.20050402160141.02f22b70@mail.telecommunity.com> <5.1.1.6.0.20050403102214.026609f0@mail.telecommunity.com> Message-ID: <20050404182620.GD16567@ActiveState.com> [Phillip J. Eby wrote] > At 08:14 AM 4/3/05 +0200, Vivian De Smedt wrote: > >Could you confirm me that the %* isn't working on win95/win98 ? > > test.bat: > @echo off > echo %* > > Running 'test test' Outputs an '*', not any arguments. So no, it doesn't > work. Here is what the Perl world does with there perl2bat conversion: @rem = '--*-Perl-*-- @echo off if "%OS%" == "Windows_NT" goto WinNT perl -x -S "%0" %1 %2 %3 %4 %5 %6 %7 %8 %9 goto endofperl :WinNT perl -x -S %0 %* if NOT "%COMSPEC%" == "%SystemRoot%\system32\cmd.exe" goto endofperl if %errorlevel% == 9009 echo You do not have Perl in your PATH. if errorlevel 1 goto script_failed_so_exit_with_non_zero_val 2>nul goto endofperl @rem '; #!perl #line 15 eval 'exec C:\Perl58\bin\perl.exe -S $0 ${1+"$@"}' if $running_under_some_shell; ... Obviously we can't do the kind of script-as-a-batch-file wrapping that Perl can get away with with its @array_variable_name syntax, but there are some good ideas here: - If one *is* using a WinNT-flavour Windows then the "%*" mechanism gets used, otherwise it gracefully falls back to the explicit "%1 %2 %3 ..." for Win9x-flavours. - If on WinNT *and* the COMSPEC is cmd.exe then the %ERRORLEVEL% 9009 code is used to deal with "perl" (in our case "python") not having properly been found. Cheers, Trent -- Trent Mick TrentM@ActiveState.com From calvin at users.sourceforge.net Mon Apr 4 20:49:11 2005 From: calvin at users.sourceforge.net (Bastian Kleineidam) Date: Mon Apr 4 20:49:12 2005 Subject: [Distutils] small improvement of the script functionality under win32 In-Reply-To: <20050404182620.GD16567@ActiveState.com> References: <5.1.1.6.0.20050402160141.02f22b70@mail.telecommunity.com> <5.1.1.6.0.20050402144559.03b50700@mail.telecommunity.com> <5.1.1.6.0.20050402030427.02a01950@mail.telecommunity.com> <424E3DAD.10306@vdesmedt.com> <5.1.1.6.0.20050402030427.02a01950@mail.telecommunity.com> <5.1.1.6.0.20050402144559.03b50700@mail.telecommunity.com> <5.1.1.6.0.20050402160141.02f22b70@mail.telecommunity.com> <5.1.1.6.0.20050403102214.026609f0@mail.telecommunity.com> <20050404182620.GD16567@ActiveState.com> Message-ID: <20050404184911.GA12945@treasure> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, On Mon, Apr 04, 2005 at 11:26:21AM -0700, Trent Mick wrote: > - If one *is* using a WinNT-flavour Windows then the "%*" mechanism gets > used, otherwise it gracefully falls back to the explicit "%1 %2 %3 > ..." for Win9x-flavours. I wanted to note that %* expands to an unlimited number of arguments, not just ten. Only the direct access (which you have to use under Win9x systems) is limited to the first ten with the %0-9 scheme. Regards, Bastian - -- ,''`. Bastian Kleineidam : :' : GnuPG Schl?ssel `. `' gpg --keyserver wwwkeys.pgp.net --recv-keys 32EC6F3E `- -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.0 (GNU/Linux) iD8DBQFCUYwneBwlBDLsbz4RAtgvAJ4/XCIl2QFb1eAioCfbxmv+asbkGgCffOuQ A7wbjl4YRrK0iFQVTV07c4U= =+FPJ -----END PGP SIGNATURE----- From pje at telecommunity.com Mon Apr 4 20:53:04 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon Apr 4 20:50:32 2005 Subject: [Distutils] small improvement of the script functionality under win32 In-Reply-To: <20050404182620.GD16567@ActiveState.com> References: <5.1.1.6.0.20050403102214.026609f0@mail.telecommunity.com> <5.1.1.6.0.20050402160141.02f22b70@mail.telecommunity.com> <5.1.1.6.0.20050402144559.03b50700@mail.telecommunity.com> <5.1.1.6.0.20050402030427.02a01950@mail.telecommunity.com> <424E3DAD.10306@vdesmedt.com> <5.1.1.6.0.20050402030427.02a01950@mail.telecommunity.com> <5.1.1.6.0.20050402144559.03b50700@mail.telecommunity.com> <5.1.1.6.0.20050402160141.02f22b70@mail.telecommunity.com> <5.1.1.6.0.20050403102214.026609f0@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20050404144219.027009d0@mail.telecommunity.com> At 11:26 AM 4/4/05 -0700, Trent Mick wrote: >[Phillip J. Eby wrote] > > At 08:14 AM 4/3/05 +0200, Vivian De Smedt wrote: > > >Could you confirm me that the %* isn't working on win95/win98 ? > > > > test.bat: > > @echo off > > echo %* > > > > Running 'test test' Outputs an '*', not any arguments. So no, it doesn't > > work. > >Here is what the Perl world does with there perl2bat conversion: > > @rem = '--*-Perl-*-- > @echo off > if "%OS%" == "Windows_NT" goto WinNT > perl -x -S "%0" %1 %2 %3 %4 %5 %6 %7 %8 %9 > goto endofperl > :WinNT > perl -x -S %0 %* > if NOT "%COMSPEC%" == "%SystemRoot%\system32\cmd.exe" goto endofperl > if %errorlevel% == 9009 echo You do not have Perl in your PATH. > if errorlevel 1 goto script_failed_so_exit_with_non_zero_val 2>nul > goto endofperl > @rem '; Hm. You know, with 'python -x' you could render the above as something like: @echo off rem = """ pathtopython -x scriptfilenamehere %0 %1 ... goto finished """ [program code goes here] """ :finished rem """ Unfortunately, you have to embed the actual filename because Python doesn't have an equivalent to Perl's -S. Anyway, this isn't really a workable solution; I think creating a separate .bat file is still the best way to go. From mal at egenix.com Mon Apr 4 22:46:52 2005 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Apr 4 22:47:01 2005 Subject: [Distutils] small improvement of the script functionality under win32 In-Reply-To: <5.1.1.6.0.20050404144219.027009d0@mail.telecommunity.com> References: <5.1.1.6.0.20050403102214.026609f0@mail.telecommunity.com> <5.1.1.6.0.20050402160141.02f22b70@mail.telecommunity.com> <5.1.1.6.0.20050402144559.03b50700@mail.telecommunity.com> <5.1.1.6.0.20050402030427.02a01950@mail.telecommunity.com> <424E3DAD.10306@vdesmedt.com> <5.1.1.6.0.20050402030427.02a01950@mail.telecommunity.com> <5.1.1.6.0.20050402144559.03b50700@mail.telecommunity.com> <5.1.1.6.0.20050402160141.02f22b70@mail.telecommunity.com> <5.1.1.6.0.20050403102214.026609f0@mail.telecommunity.com> <5.1.1.6.0.20050404144219.027009d0@mail.telecommunity.com> Message-ID: <4251A7BC.5080002@egenix.com> [Lots of different techniques to trick Windows/DOS into starting a Python script] It is becoming rather obvious that the python -x trick is not sufficient for Windows/DOS. Wouldn't it be a lot cleaner if we invented a new command line switch to provide whatever parsing technique is necessary in order to provide a truely clean solution with everything in one file ? E.g. perhaps we need something like: -X n : skip the first n lines in support of OS specific startup code And then: @echo off pathtopython -X 4 scriptfilenamehere %0 %1 ... exit # Start of Python code [program code goes here] -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 04 2005) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From niepraschk at chbrb.berlin.ptb.de Mon Apr 4 22:33:15 2005 From: niepraschk at chbrb.berlin.ptb.de (niepraschk@chbrb.berlin.ptb.de) Date: Mon Apr 4 23:18:28 2005 Subject: [Distutils] HELLO Message-ID: <20050404211827.564391E4003@bag.python.org> The message contains Unicode characters and has been sent as a binary attachment. From trentm at ActiveState.com Mon Apr 4 23:55:21 2005 From: trentm at ActiveState.com (Trent Mick) Date: Tue Apr 5 00:00:01 2005 Subject: [Distutils] small improvement of the script functionality under win32 In-Reply-To: <4251A7BC.5080002@egenix.com> References: <5.1.1.6.0.20050402160141.02f22b70@mail.telecommunity.com> <5.1.1.6.0.20050402144559.03b50700@mail.telecommunity.com> <5.1.1.6.0.20050402030427.02a01950@mail.telecommunity.com> <424E3DAD.10306@vdesmedt.com> <5.1.1.6.0.20050402030427.02a01950@mail.telecommunity.com> <5.1.1.6.0.20050402144559.03b50700@mail.telecommunity.com> <5.1.1.6.0.20050402160141.02f22b70@mail.telecommunity.com> <5.1.1.6.0.20050403102214.026609f0@mail.telecommunity.com> <5.1.1.6.0.20050404144219.027009d0@mail.telecommunity.com> <4251A7BC.5080002@egenix.com> Message-ID: <20050404215521.GC25486@ActiveState.com> [M.-A. Lemburg wrote] > [Lots of different techniques to trick Windows/DOS into starting > a Python script] > > It is becoming rather obvious that the python -x trick is not > sufficient for Windows/DOS. I didn't even know that "-x" existed or how to use it properly. It defeats my searching abilities to find documentation for it. > E.g. perhaps we need something like: > > -X n : skip the first n lines in support of OS specific startup code > > And then: > > @echo off > pathtopython -X 4 scriptfilenamehere %0 %1 ... > exit > # Start of Python code > [program code goes here] Sounds useful. Note that, until now, I hadn't really grokked the point of this thread :). I was just throwing in my experiences of the Windows shell. I don't use the distutils script handling stuff at all, so am not a good person to make a call on doing this. I just saw mention of effbot's "exemaker" today: http://effbot.org/zone/exemaker.htm I *think* that having something like this would solve the problem. I've done something similar to exemaker (i.e. create little stub launcher executable for Python scripts that I distribute). I've preferred using an executable launcher stub to using a batch file partially because of the .bat syntax differences between Win9x and WinNT, but also because sometimes when breaking a process started by a batch file with Ctrl+C the shell will spit out "Would you like to stop the batch file also?" -- or something along those lines. I don't know if adding something like exemaker would be feasible or even desirable: - might cause other headaches - people might not want a launcher file separate from the script file itself One benefit of a separate launcher file over a script-in-batchfile-clothing approach: we might be able to take advantage of python's "-m" option such that a Python script that can also operate as a module (e.g. timeit.py, pydoc.py) could stay in the Lib dir and the launch could be put in the bindir. Trent -- Trent Mick TrentM@ActiveState.com From brian_dunfordshore at bridge.com Mon Apr 4 23:09:31 2005 From: brian_dunfordshore at bridge.com (brian_dunfordshore@bridge.com) Date: Tue Apr 5 04:03:29 2005 Subject: [Distutils] STATUS Message-ID: <20050405020328.0C79F1E4006@bag.python.org> The message contains Unicode characters and has been sent as a binary attachment. From giles at cvs.ghostscript.com Mon Apr 4 23:44:54 2005 From: giles at cvs.ghostscript.com (giles@cvs.ghostscript.com) Date: Tue Apr 5 05:06:10 2005 Subject: [Distutils] (no subject) Message-ID: <20050405030608.D4A0F1E4006@bag.python.org> The message contains Unicode characters and has been sent as a binary attachment. From postmaster at gdds.com Tue Apr 5 06:40:38 2005 From: postmaster at gdds.com (GDDS Postmaster AZ25EGE01) Date: Tue Apr 5 06:50:59 2005 Subject: [Distutils] Mail could not be delivered Message-ID: <20050405045058.421361E4003@bag.python.org> ****** Message from InterScan Messaging Security Suite ****** Sent <<< RCPT TO: Received >>> 550 pomeranz@netcom.com...user unknown Unable to deliver message to (and other recipients in the same domain). ************************ End of message ********************** -------------- next part -------------- An embedded message was scrubbed... From: distutils-sig@python.org Subject: Status Date: Mon, 4 Apr 2005 13:36:29 -0700 Size: 859 Url: http://mail.python.org/pipermail/distutils-sig/attachments/20050404/980fe97c/attachment.mht From postmaster at gdds.com Tue Apr 5 03:05:02 2005 From: postmaster at gdds.com (GDDS Postmaster AZ25EGE02) Date: Tue Apr 5 06:54:02 2005 Subject: [Distutils] Mail could not be delivered Message-ID: <20050405045401.24C101E4003@bag.python.org> ****** Message from InterScan Messaging Security Suite ****** Sent <<< RCPT TO: Received >>> 550 sender verify failed Unable to deliver message to (and other recipients in the same domain). ************************ End of message ********************** -------------- next part -------------- An embedded message was scrubbed... From: distutils-sig@python.org Subject: HELLO Date: Mon, 4 Apr 2005 13:46:41 -0700 Size: 867 Url: http://mail.python.org/pipermail/distutils-sig/attachments/20050404/f33b0fa4/attachment-0001.mht From ctichler at cloud9.net Mon Apr 4 22:36:04 2005 From: ctichler at cloud9.net (ctichler@cloud9.net) Date: Tue Apr 5 09:06:32 2005 Subject: [Distutils] gnotenwgtdvhq Message-ID: <20050405070631.771CF1E4006@bag.python.org> The message contains Unicode characters and has been sent as a binary attachment. From kde-i18n-tr-bounces at kde.org Tue Apr 5 10:33:02 2005 From: kde-i18n-tr-bounces at kde.org (kde-i18n-tr-bounces@kde.org) Date: Tue Apr 5 10:33:04 2005 Subject: [Distutils] The results of your email commands Message-ID: The results of your email command are provided below. Attached is your original message. - Results: Ignoring non-text/plain MIME parts - Done. -------------- next part -------------- An embedded message was scrubbed... From: distutils-sig@python.org Subject: hello Date: Mon, 4 Apr 2005 13:39:14 -0700 Size: 1731 Url: http://mail.python.org/pipermail/distutils-sig/attachments/20050405/50aa4cc2/attachment.mht From postmaster at gdds.com Tue Apr 5 11:07:08 2005 From: postmaster at gdds.com (GDDS Postmaster AZ25EGE02) Date: Tue Apr 5 11:43:32 2005 Subject: [Distutils] Mail could not be delivered Message-ID: <20050405094331.884B91E4004@bag.python.org> ****** Message from InterScan Messaging Security Suite ****** Sent <<< RCPT TO: Received >>> 550 sender verify failed Unable to deliver message to (and other recipients in the same domain). ************************ End of message ********************** -------------- next part -------------- An embedded message was scrubbed... From: distutils-sig@python.org Subject: Error Date: Mon, 4 Apr 2005 14:47:04 -0700 Size: 890 Url: http://mail.python.org/pipermail/distutils-sig/attachments/20050405/ef55d928/attachment.mht From p.f.moore at gmail.com Tue Apr 5 12:44:19 2005 From: p.f.moore at gmail.com (Paul Moore) Date: Tue Apr 5 12:44:21 2005 Subject: [Distutils] small improvement of the script functionality under win32 In-Reply-To: <20050404215521.GC25486@ActiveState.com> References: <424E3DAD.10306@vdesmedt.com> <5.1.1.6.0.20050402030427.02a01950@mail.telecommunity.com> <5.1.1.6.0.20050402144559.03b50700@mail.telecommunity.com> <5.1.1.6.0.20050402160141.02f22b70@mail.telecommunity.com> <5.1.1.6.0.20050403102214.026609f0@mail.telecommunity.com> <5.1.1.6.0.20050404144219.027009d0@mail.telecommunity.com> <4251A7BC.5080002@egenix.com> <20050404215521.GC25486@ActiveState.com> Message-ID: <79990c6b05040503443763ddf2@mail.gmail.com> On Apr 4, 2005 10:55 PM, Trent Mick wrote: > I don't know if adding something like exemaker would be feasible or even > desirable: > - might cause other headaches > - people might not want a launcher file separate from the script > file itself To be honest, I think that personal preference issues are a big problem here. I find the "Do you want to terminate the batch file" message annoying, but I have no problem with a .py extension. I have .py registered in PATHEXT and all the necessary places to make .py files "executable" - not all of the necessary things are done by the Python installer, though (for reasons which I accept) and so this isn't a universally acceptable solution either. > One benefit of a separate launcher file over a > script-in-batchfile-clothing approach: we might be able to take > advantage of python's "-m" option such that a Python script that can > also operate as a module (e.g. timeit.py, pydoc.py) could stay in the > Lib dir and the launch could be put in the bindir. To be honest, the -m option (once support for packages appears, which I believe is scheduled for 2.5) probably makes script files 99% unnecessary. For people who absolutely insist on a "command", a 1-liner .bat file @pathtopython -m module %1 %2 %3 %4 %5 %6 %7 %8 %9 (avoid %* for command.com compatibility) should be adequate. With distutils support, a suitable platform-specific script like this could even be generated automatically, if needed. This is mainly a social change, though - persuading package maintainers to write their code such that -m can be used. Maybe a documentation change is the best approach. Add some text to section 2.4 (Installing Scripts) of "Distributing Python Modules" to say something like: """ By their nature, script files are inherently platform dependent - certain platforms require particular file extensions, or other conventions. Since Python 2.4, the Python executable has supported a command-line option -m (reference to the docs here) which allows modules to contain an executable section (see, for example, the timeit or compileall library modules). Where possible, writing modules which support the use of -m will be more portable than distributing standalone scripts. """ Paul. From yszhao at ac-tech.com Tue Apr 5 21:50:40 2005 From: yszhao at ac-tech.com (Yongsheng Zhao) Date: Tue Apr 5 21:50:27 2005 Subject: [Distutils] Installation error Message-ID: <48543B17A501F44F9361395023F70D5DC27036@act-mail.actech.int> Hello, When I installed the package, I got the following error: # python setup.py install Traceback (most recent call last): File "setup.py", line 30, in ? packages = ['distutils', 'distutils.command'], File "/tmp/Distutils-1.0.2/distutils/core.py", line 101, in setup _setup_distribution = dist = klass(attrs) File "/tmp/Distutils-1.0.2/distutils/dist.py", line 130, in __init__ setattr(self, method_name, getattr(self.metadata, method_name)) AttributeError: DistributionMetadata instance has no attribute 'get___doc__' Anyone know what happens? Thanks. Regards Yongsheng Zhao From musenik at yahoo.com Tue Apr 12 18:12:26 2005 From: musenik at yahoo.com (Keith Nemitz) Date: Tue Apr 12 18:12:28 2005 Subject: [Distutils] py2exe makes pygame.sysFont crash In-Reply-To: <20050406100004.85AEF1E400D@bag.python.org> Message-ID: <20050412161226.9600.qmail@web81707.mail.yahoo.com> I have a small game that runs fine on Windows XP with just the interpreter, but after using py2exe to create an executable, the new exe crashes with this error message: ------------------------ Pygame Parachute Traceback: File "pygame\sysfont.pyc", line 253, in SysFont Fatal Python error: (pygame parachute) Segmentation Fault This application has requested the Runtime to terminate it in an unusual way. Please contact the application's support team for more information. ---------------------------------------------- I'm using python 2.3 and pygame 1.6. I assume that I need to explicitly add a library, but I'm not sure what or how. I hope someone might be able to assist me. Here is the py2exe code: ---------------------------------------------- import time import sys # ModuleFinder can't handle runtime changes to __path__, but win32com uses them try: import modulefinder import win32com for p in win32com.__path__[1:]: modulefinder.AddPackagePath("win32com", p) for extra in ["win32com.shell"]: #,"win32com.mapi" __import__(extra) m = sys.modules[extra] for p in m.__path__[1:]: modulefinder.AddPackagePath(extra, p) except ImportError: # no build path setup, no worries. raise RuntimeError("no can load win32com?") pass from distutils.core import setup import py2exe setup( # The first three parameters are not required, if at least a # 'version' is given, then a versioninfo resource is built from # them and added to the executables. version = "0.2.01", description = "py2exe script", name = "The Witch's Yarn", # targets to build console = ["main.py"], #windows = ["main.py"], ) From theller at python.net Tue Apr 12 19:52:54 2005 From: theller at python.net (Thomas Heller) Date: Tue Apr 12 19:54:41 2005 Subject: [Distutils] Re: py2exe makes pygame.sysFont crash References: <20050406100004.85AEF1E400D@bag.python.org> <20050412161226.9600.qmail@web81707.mail.yahoo.com> Message-ID: Keith Nemitz writes: > I have a small game that runs fine on Windows XP with > just the interpreter, but after using py2exe to create > an executable, the new exe crashes with this error > message: > > ------------------------ > Pygame Parachute Traceback: > File "pygame\sysfont.pyc", line 253, in SysFont > Fatal Python error: (pygame parachute) Segmentation > Fault > > This application has requested the Runtime to > terminate it in an unusual way. > Please contact the application's support team for more > information. > ---------------------------------------------- > > I'm using python 2.3 and pygame 1.6. I assume that I > need to explicitly add a library, but I'm not sure > what or how. I hope someone might be able to assist > me. Here is the py2exe code: First possibility: when you run the game in the interpreter with 'python -v mygame.py' python will print which modules are imported. Second: run 'python -i mygame.py', which will enter the interactive interpreter after the script finishes. Then you can inspect 'sys.modules' to find out what has been imported. The most important next step, if that has helped, would be to either notify the pygame developers to update their py2exe sample script, and/or write a helpful page in the py2exe wiki. HTH, Thomas From jeremy at jeremysanders.net Wed Apr 13 19:24:29 2005 From: jeremy at jeremysanders.net (Jeremy Sanders) Date: Wed Apr 13 19:24:47 2005 Subject: [Distutils] package_data issue (chopping off characters) Message-ID: I have a strange problem with distutils and package_data. I'm using python 2.3.3, with the setuptools from CVS (so that I can use package_data). My setup.py contains: ------------------------------------------------------------------------ #!/usr/bin/env python # Veusz distutils setup script # $Id: setup.py,v 1.2 2005/04/13 17:03:55 jeremysanders Exp $ import glob from setuptools import setup version = open('VERSION').read().strip() descr = '''Veusz is a scientific plotting package, designed to create publication-ready Postscript output. It features GUI, command-line, and scripting interfaces. Graphs are constructed from "widgets", allowing complex layouts to be designed. Veusz supports plotting functions, data with errors, keys, labels, stacked plots, multiple plots, and fitting data.''' setup(name = 'veusz', version = version, description = 'A scientific plotting package', long_description = descr, author = 'Jeremy Sanders', author_email = 'jeremy@jeremysanders.net', url = 'http://home.gna.org/veusz/', license = 'GPL', classifiers = ['Programming Language :: Python', 'Development Status :: 4 - Beta', 'Environment :: X11 Applications :: Qt', 'Intended Audiance :: Advanced End Users', 'Licence :: OSI Approved :: ' 'GNU General Public License (GPL)', 'Topic :: Scientific/Engineering :: Visualization'], package_dir = { 'veusz': '', 'veusz.dialogs': 'dialogs', 'veusz.document': 'document', 'veusz.setting': 'setting', 'veusz.utils': 'utils', 'veusz.widgets': 'widgets', 'veusz.windows': 'windows' }, package_data = { 'veusz.windows': ['icons/*.png'], 'veusz': ['images/*.png'] }, packages = ['veusz', 'veusz.dialogs', 'veusz.document', 'veusz.setting', 'veusz.utils', 'veusz.widgets', 'veusz.windows'] ) ------------------------------------------------------------ When it is run with python setup.py bdist_rpm I get, running bdist running bdist_dumb running build running build_py creating build/lib/veusz/mages error: can't copy 'mages/logo.png': doesn't exist or not a regular file (I have tried removing the build and dist directories). Somehow the first character of 'images' has been chopped off. If I remove the package_data command, everything works (except the data don't get packaged). Any ideas??? Thanks Jeremy -- Jeremy Sanders http://www.jeremysanders.net/ Cambridge, UK Public Key Server PGP Key ID: E1AAE053 From seefeld at sympatico.ca Wed Apr 13 15:45:11 2005 From: seefeld at sympatico.ca (Stefan Seefeld) Date: Wed Apr 13 19:45:59 2005 Subject: [Distutils] package_data issue (chopping off characters) In-Reply-To: References: Message-ID: <425D2267.2020101@sympatico.ca> Jeremy Sanders wrote: > When it is run with > python setup.py bdist_rpm > > I get, > > running bdist > running bdist_dumb > running build > running build_py > creating build/lib/veusz/mages > error: can't copy 'mages/logo.png': doesn't exist or not a regular file Coincidentally I have seen the same symptom last night, though the context was different: The bdist_rpm command calls the 'install' commands (with all its subcommands) to install everything under a predefined root directory. It tells the 'install' command about that root path and the install command modifies all its paths accordingly (search for 'change_root' in distutils/commands/install.py). However, I made a small error in my 'install_clib' command's 'get_outputs()' method in that it didn't consider the modified paths but the original ones. bdist_rpm then cut off what it thought was the root directory, but due to the error the resulting paths were incorrect (i.e. appeared 'chopped off' as in your case). You didn't mention any custom commands, and so I'm not sure that's really related to your error. It may give some hints, though, as to where to look further. HTH, Stefan From jeremy at jeremysanders.net Thu Apr 14 11:58:49 2005 From: jeremy at jeremysanders.net (Jeremy Sanders) Date: Thu Apr 14 11:59:00 2005 Subject: [Distutils] package_data issue (chopping off characters) In-Reply-To: <425D2267.2020101@sympatico.ca> References: <425D2267.2020101@sympatico.ca> Message-ID: On Wed, 13 Apr 2005, Stefan Seefeld wrote: > Coincidentally I have seen the same symptom last night, though the > context was different: > > The bdist_rpm command calls the 'install' commands (with all its > subcommands) to install everything under a predefined root directory. It > tells the 'install' command about that root path and the install command > modifies all its paths accordingly (search for 'change_root' in > distutils/commands/install.py). > > However, I made a small error in my 'install_clib' command's > 'get_outputs()' method in that it didn't consider the modified paths but > the original ones. bdist_rpm then cut off what it thought was the root > directory, but due to the error the resulting paths were incorrect (i.e. > appeared 'chopped off' as in your case). > > You didn't mention any custom commands, and so I'm not sure that's > really related to your error. It may give some hints, though, as to > where to look further. No, I haven't used any custom commands. If anyone would like to look at the problem, download http://download.gna.org/veusz/testing/veusz-pre0.5.tar.gz unpack, cd to veusz, and run python setup.py bdist It then gives an error: copying windows/consolewindow.py -> build/lib/veusz/windows copying windows/__init__.py -> build/lib/veusz/windows copying windows/mainwindow.py -> build/lib/veusz/windows copying windows/treeeditwindow.py -> build/lib/veusz/windows copying windows/simplewindow.py -> build/lib/veusz/windows copying windows/action.py -> build/lib/veusz/windows copying windows/plotwindow.py -> build/lib/veusz/windows error: can't copy 'ERSION': doesn't exist or not a regular file where the setup.py file refers to VERSION, not ERSION. Thanks Jeremy -- Jeremy Sanders http://www.jeremysanders.net/ Cambridge, UK Public Key Server PGP Key ID: E1AAE053 From calvin at users.sourceforge.net Thu Apr 14 14:21:34 2005 From: calvin at users.sourceforge.net (Bastian Kleineidam) Date: Thu Apr 14 14:21:35 2005 Subject: [Distutils] package_data issue (chopping off characters) In-Reply-To: References: <425D2267.2020101@sympatico.ca> Message-ID: <20050414122134.GA30392@treasure> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi Jeremy, On Thu, Apr 14, 2005 at 10:58:49AM +0100, Jeremy Sanders wrote: > http://download.gna.org/veusz/testing/veusz-pre0.5.tar.gz > unpack, cd to veusz, and run > python setup.py bdist [...] > error: can't copy 'ERSION': doesn't exist or not a regular file You are not using an official distutils build, but an internal (modified?) version "setuptools". Please use "from distutils.core import setup" instead of "from setuptools import setup" Then it works. Regards, Bastian - -- ,''`. Bastian Kleineidam : :' : GnuPG Schl?ssel `. `' gpg --keyserver wwwkeys.pgp.net --recv-keys 32EC6F3E `- -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.0 (GNU/Linux) iD8DBQFCXmBOeBwlBDLsbz4RArrAAJ9rgslNIXXjcMsR8RrFL8f0xJXfpgCfZnRd kCjcr9sG6XNEE4wR9hOibtQ= =axu9 -----END PGP SIGNATURE----- From jeremy at jeremysanders.net Thu Apr 14 15:37:49 2005 From: jeremy at jeremysanders.net (Jeremy Sanders) Date: Thu Apr 14 15:37:56 2005 Subject: [Distutils] package_data issue (chopping off characters) In-Reply-To: <425D2267.2020101@sympatico.ca> References: <425D2267.2020101@sympatico.ca> Message-ID: On Wed, 13 Apr 2005, Stefan Seefeld wrote: > Jeremy Sanders wrote: > >> When it is run with >> python setup.py bdist_rpm >> >> I get, >> >> running bdist >> running bdist_dumb >> running build >> running build_py >> creating build/lib/veusz/mages >> error: can't copy 'mages/logo.png': doesn't exist or not a regular file Apologies to Bastian Kleineidam, I'm not on the list so this isn't a direct reply to your email! I'm using the cvs version "setuptools", as it was suggested in an email on this list: http://mail.python.org/pipermail/distutils-sig/2004-October/004214.html as I wanted my setup.py to work under Python-2.3. Are the distutils from Python-2.4 likely to work if I transplant it into my own program?? Thanks Jeremy -- Jeremy Sanders http://www.jeremysanders.net/ Cambridge, UK Public Key Server PGP Key ID: E1AAE053 From jeremy at jeremysanders.net Thu Apr 14 16:08:48 2005 From: jeremy at jeremysanders.net (Jeremy Sanders) Date: Thu Apr 14 16:08:53 2005 Subject: [Distutils] package_data issue (chopping off characters) In-Reply-To: <425D2267.2020101@sympatico.ca> References: <425D2267.2020101@sympatico.ca> Message-ID: Sorry for not replying properly in the thread, but I think I've found the bug. I downloaded python from cvs and took the latest distutils from there. In command/build_py.py there is a function: def get_data_files (self): """Generate list of '(package,src_dir,build_dir,filenames)' tuples""" data = [] if not self.packages: return data for package in self.packages: # Locate package source directory src_dir = self.get_package_dir(package) # Compute package build directory build_dir = os.path.join(*([self.build_lib] + package.split('.'))) # Length of path to strip from found files plen = len(src_dir)+1 # Strip directory from globbed filenames filenames = [ file[plen:] for file in self.find_data_files(package, src_dir) ] data.append((package, src_dir, build_dir, filenames)) return data Note that the filename is chopped using file[plen:] above. If src_dir=="", as I have in my setup.py file (which is a perfectly acceptable package directory), then plen == 1, and so the first character gets chopped from each of the file names. self.find_data_files returns the correct files, but the first character gets chopped from them. Jeremy -- Jeremy Sanders http://www.jeremysanders.net/ Cambridge, UK Public Key Server PGP Key ID: E1AAE053 From wurst2 at kampfwurst.net Thu Apr 14 16:33:22 2005 From: wurst2 at kampfwurst.net (Bastian Kleineidam) Date: Thu Apr 14 16:33:22 2005 Subject: [Distutils] package_data issue (chopping off characters) In-Reply-To: References: <425D2267.2020101@sympatico.ca> Message-ID: <20050414143322.GA529@treasure> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, On Thu, Apr 14, 2005 at 02:37:49PM +0100, Jeremy Sanders wrote: > I'm using the cvs version "setuptools", as it was suggested in an email on > this list: > > http://mail.python.org/pipermail/distutils-sig/2004-October/004214.html > > as I wanted my setup.py to work under Python-2.3. Ah, my fault. I tested only with 2.3. With 2.4 I can reproduce the error. I attached a quick fix that you can try out. If it works for you, I will submit it to the patch manager. > Are the distutils from Python-2.4 likely to work if I transplant it into > my own program?? The 2.4 distutils should also work under Python 2.3. Regards, Bastian PS: There are still some 'distutils' imports in your setuptools/ directory. You should change them also to 'setuptools' just to be sure that only those methods in setuptools are used. - -- ,''`. Bastian Kleineidam : :' : GnuPG Schl?ssel `. `' gpg --keyserver wwwkeys.pgp.net --recv-keys 32EC6F3E `- -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.0 (GNU/Linux) iD8DBQFCXn8yeBwlBDLsbz4RAurXAJ0QqBSV2S/M3SJPyVpVm9r0EXcMtwCdEtgL 4XzoWZrCcC5ysu7D97pF5UY= =qj+h -----END PGP SIGNATURE----- From calvin at users.sourceforge.net Thu Apr 14 16:35:44 2005 From: calvin at users.sourceforge.net (Bastian Kleineidam) Date: Thu Apr 14 16:35:44 2005 Subject: [Distutils] package_data chopping off character bugfix Message-ID: <20050414143544.GA589@treasure> Skipped content of type multipart/mixed-------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://mail.python.org/pipermail/distutils-sig/attachments/20050414/986844dd/attachment.pgp From jeremy at jeremysanders.net Thu Apr 14 21:57:22 2005 From: jeremy at jeremysanders.net (Jeremy Sanders) Date: Thu Apr 14 21:57:28 2005 Subject: [Distutils] Error on CVS Message-ID: [jss@serv veusz]$ python setup.py bdist_rpm running bdist_rpm creating build creating build/bdist.linux-i686 creating build/bdist.linux-i686/rpm creating build/bdist.linux-i686/rpm/SOURCES creating build/bdist.linux-i686/rpm/SPECS creating build/bdist.linux-i686/rpm/BUILD creating build/bdist.linux-i686/rpm/RPMS creating build/bdist.linux-i686/rpm/SRPMS writing 'build/bdist.linux-i686/rpm/SPECS/veusz.spec' Traceback (most recent call last): File "setup.py", line 62, in ? packages = ['veusz', File "/data/home/jss/code/veusz/veusz/distutils/core.py", line 151, in setup dist.run_commands() File "/data/home/jss/code/veusz/veusz/distutils/dist.py", line 974, in run_commands self.run_command(cmd) File "/data/home/jss/code/veusz/veusz/distutils/dist.py", line 994, in run_command cmd_obj.run() File "/data/home/jss/code/veusz/veusz/distutils/command/bdist_rpm.py", line 301, in run saved_dist_files = self.distributuion.dist_files[:] File "/data/home/jss/code/veusz/veusz/distutils/cmd.py", line 112, in __getattr__ raise AttributeError, attr AttributeError: distributuion Looks like a typo! Here's a patch. --- command/bdist_rpm.py~ 2005-03-28 02:05:48.000000000 +0100 +++ command/bdist_rpm.py 2005-04-14 20:56:52.500042048 +0100 @@ -298,7 +298,7 @@ # Make a source distribution and copy to SOURCES directory with # optional icon. - saved_dist_files = self.distributuion.dist_files[:] + saved_dist_files = self.distribution.dist_files[:] sdist = self.reinitialize_command('sdist') if self.use_bzip2: sdist.formats = ['bztar'] Jeremy -- Jeremy Sanders http://www.jeremysanders.net/ Cambridge, UK Public Key Server PGP Key ID: E1AAE053 From anthony at interlink.com.au Fri Apr 15 08:23:31 2005 From: anthony at interlink.com.au (Anthony Baxter) Date: Fri Apr 15 08:23:57 2005 Subject: [Distutils] Error on CVS In-Reply-To: References: Message-ID: <200504151623.31978.anthony@interlink.com.au> On Friday 15 April 2005 05:57, Jeremy Sanders wrote: > saved_dist_files = self.distributuion.dist_files[:] > File "/data/home/jss/code/veusz/veusz/distutils/cmd.py", line 112, in > __getattr__ > raise AttributeError, attr > AttributeError: distributuion Thanks! Fixed in CVS as of r1.52. Anthony -- Anthony Baxter It's never too late to have a happy childhood. From cludwig at cdc.informatik.tu-darmstadt.de Fri Apr 15 15:09:26 2005 From: cludwig at cdc.informatik.tu-darmstadt.de (Christoph Ludwig) Date: Fri Apr 15 15:09:29 2005 Subject: [Distutils] [build_ext] more than one external library Message-ID: <20050415130926.GC2724@lap200.cdc.informatik.tu-darmstadt.de> Hi, I use the distutils shipped with Python 2.4 on a i686-pc-linux-gnu system. I am wondering how can I pass more than one external library to distutils' build_ext option (on the command line or even better via setup.cfg)? In analogy to --include-dirs I'd expect ./setup.py build_ext --libraries A:B to generate a linker call similar to: g++ -lA -lB # ... other arguments and options But I get: g++ -lA:B # ... I also tried ./setup.py build_ext --libraries 'A B' ./setup.py build_ext --libraries A B ./setup.py build_ext --libraries A,B ./setup.py build_ext --libraries A --libraries B but to no avail. (In the last case the second --libraries option seems to overwrite the first one.) Is this feature missing and I need to put the library list in the libraries option of the respective Extension constructor in setup.py? Or did I fail to try the correct syntax? Best regards Christoph -- http://www.informatik.tu-darmstadt.de/TI/Mitarbeiter/cludwig.html LiDIA: http://www.informatik.tu-darmstadt.de/TI/LiDIA/Welcome.html From jeremy at jeremysanders.net Fri Apr 15 19:10:49 2005 From: jeremy at jeremysanders.net (Jeremy Sanders) Date: Fri Apr 15 19:10:57 2005 Subject: [Distutils] bdist_rpm and bdist on x86-64 Message-ID: setup.py bdist and bdist_rpm appear to fail on my x86-64 machine running Fedora Core 2. ... running build_scripts creating build/scripts-2.3 copying and adjusting scripts/veusz -> build/scripts-2.3 copying and adjusting scripts/veusz_listen -> build/scripts-2.3 changing mode of build/scripts-2.3/veusz from 664 to 775 changing mode of build/scripts-2.3/veusz_listen from 664 to 775 installing to build/bdist.linux-x86_64/dumb running install error: invalid Python installation: unable to open /usr/lib/python2.3/config/Makefile (No such file or directory) It appears that distutils is looking in /usr/lib/python2.3/config rather than /usr/lib64/python2.3/config I haven't been able to easily locate where it's going wrong. Any ideas?? I'm using the distutils from Python-2.4.1 on Python-2.3.3 (64 bit). Thanks Jeremy -- Jeremy Sanders http://www.jeremysanders.net/ Cambridge, UK Public Key Server PGP Key ID: E1AAE053 From mal at egenix.com Fri Apr 15 19:44:28 2005 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Apr 15 19:44:32 2005 Subject: [Distutils] bdist_rpm and bdist on x86-64 In-Reply-To: References: Message-ID: <425FFD7C.5060602@egenix.com> Jeremy Sanders wrote: > setup.py bdist and bdist_rpm appear to fail on my x86-64 machine running > Fedora Core 2. > > ... > running build_scripts > creating build/scripts-2.3 > copying and adjusting scripts/veusz -> build/scripts-2.3 > copying and adjusting scripts/veusz_listen -> build/scripts-2.3 > changing mode of build/scripts-2.3/veusz from 664 to 775 > changing mode of build/scripts-2.3/veusz_listen from 664 to 775 > installing to build/bdist.linux-x86_64/dumb > running install > error: invalid Python installation: unable to open > /usr/lib/python2.3/config/Makefile (No such file or directory) > > It appears that distutils is looking in > /usr/lib/python2.3/config > rather than > /usr/lib64/python2.3/config > > I haven't been able to easily locate where it's going wrong. Any ideas?? Check the Makefile you Python version has installed in lib/pythonX.X/config/Makefile distutils reads the specs from that file. > I'm using the distutils from Python-2.4.1 on Python-2.3.3 (64 bit). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 15 2005) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From jeremy at jeremysanders.net Fri Apr 15 21:55:05 2005 From: jeremy at jeremysanders.net (Jeremy Sanders) Date: Fri Apr 15 21:55:46 2005 Subject: [Distutils] bdist_rpm and bdist on x86-64 In-Reply-To: <425FFD7C.5060602@egenix.com> References: <425FFD7C.5060602@egenix.com> Message-ID: On Fri, 15 Apr 2005, M.-A. Lemburg wrote: > Check the Makefile you Python version has installed in > lib/pythonX.X/config/Makefile There is no /usr/lib/python2.3/config/Makefile. It is installed in /usr/lib64/python2.3/config/Makefile. That file contains # Expanded directories BINDIR= $(exec_prefix)/bin LIBDIR= $(exec_prefix)/lib64 MANDIR= /usr/share/man INCLUDEDIR= /usr/include CONFINCLUDEDIR= $(exec_prefix)/include SCRIPTDIR= $(prefix)/lib64 # Detailed destination directories BINLIBDEST= $(LIBDIR)/python$(VERSION) LIBDEST= $(SCRIPTDIR)/python$(VERSION) INCLUDEPY= $(INCLUDEDIR)/python$(VERSION) CONFINCLUDEPY= $(CONFINCLUDEDIR)/python$(VERSION) LIBP= $(LIBDIR)/python$(VERSION) So it looks like distutils is looking in the wrong place. If you look in sysconfig.py: def get_python_lib(plat_specific=0, standard_lib=0, prefix=None): """Return the directory containing the Python library (standard or site additions). If 'plat_specific' is true, return the directory containing platform-specific modules, i.e. any module from a non-pure-Python module distribution; otherwise, return the platform-shared library directory. If 'standard_lib' is true, return the directory containing standard Python library modules; otherwise, return the directory for site-specific modules. If 'prefix' is supplied, use it instead of sys.prefix or sys.exec_prefix -- i.e., ignore 'plat_specific'. """ if prefix is None: prefix = plat_specific and EXEC_PREFIX or PREFIX if os.name == "posix": libpython = os.path.join(prefix, "lib", "python" + get_python_version()) if standard_lib: return libpython else: return os.path.join(libpython, "site-packages") elif os.name == "nt": if standard_lib: return os.path.join(prefix, "Lib") else: if get_python_version() < "2.2": return prefix else: return os.path.join(PREFIX, "Lib", "site-packages") elif os.name == "mac": if plat_specific: if standard_lib: return os.path.join(prefix, "Lib", "lib-dynload") else: return os.path.join(prefix, "Lib", "site-packages") else: if standard_lib: return os.path.join(prefix, "Lib") else: return os.path.join(prefix, "Lib", "site-packages") elif os.name == "os2": if standard_lib: return os.path.join(PREFIX, "Lib") else: return os.path.join(PREFIX, "Lib", "site-packages") else: raise DistutilsPlatformError( "I don't know where Python installs its library " "on platform '%s'" % os.name) Under the posix section, distutils assumes that Python is installed in /usr/lib/python-X.X, where it's really in /usr/lib64/python-X.X. This is clearly a distutils bug. Distutils should be looking under lib64 for 64-bit x86 systems. All linux x86-64 distributions use lib64 instead of lib for 64 bit libraries. It looks like this code needs to be cleverer. Jeremy -- Jeremy Sanders http://www.jeremysanders.net/ Cambridge, UK Public Key Server PGP Key ID: E1AAE053 From mal at egenix.com Fri Apr 15 22:06:19 2005 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Apr 15 22:06:23 2005 Subject: [Distutils] bdist_rpm and bdist on x86-64 In-Reply-To: References: <425FFD7C.5060602@egenix.com> Message-ID: <42601EBB.6010009@egenix.com> Jeremy Sanders wrote: > On Fri, 15 Apr 2005, M.-A. Lemburg wrote: > >> Check the Makefile you Python version has installed in >> lib/pythonX.X/config/Makefile > > > There is no /usr/lib/python2.3/config/Makefile. It is installed in > /usr/lib64/python2.3/config/Makefile. > > That file contains > > # Expanded directories > BINDIR= $(exec_prefix)/bin > LIBDIR= $(exec_prefix)/lib64 > MANDIR= /usr/share/man > INCLUDEDIR= /usr/include > CONFINCLUDEDIR= $(exec_prefix)/include > SCRIPTDIR= $(prefix)/lib64 > > # Detailed destination directories > BINLIBDEST= $(LIBDIR)/python$(VERSION) > LIBDEST= $(SCRIPTDIR)/python$(VERSION) > INCLUDEPY= $(INCLUDEDIR)/python$(VERSION) > CONFINCLUDEPY= $(CONFINCLUDEDIR)/python$(VERSION) > LIBP= $(LIBDIR)/python$(VERSION) > > So it looks like distutils is looking in the wrong place. > > If you look in sysconfig.py: > > def get_python_lib(plat_specific=0, standard_lib=0, prefix=None): > """Return the directory containing the Python library (standard or > site additions). > > If 'plat_specific' is true, return the directory containing > platform-specific modules, i.e. any module from a non-pure-Python > module distribution; otherwise, return the platform-shared library > directory. If 'standard_lib' is true, return the directory > containing standard Python library modules; otherwise, return the > directory for site-specific modules. > > If 'prefix' is supplied, use it instead of sys.prefix or > sys.exec_prefix -- i.e., ignore 'plat_specific'. > """ > if prefix is None: > prefix = plat_specific and EXEC_PREFIX or PREFIX > > if os.name == "posix": > libpython = os.path.join(prefix, > "lib", "python" + get_python_version()) This is the problem line ^^^^^^^ > if standard_lib: > return libpython > else: > return os.path.join(libpython, "site-packages") > > elif os.name == "nt": > if standard_lib: > return os.path.join(prefix, "Lib") > else: > if get_python_version() < "2.2": > return prefix > else: > return os.path.join(PREFIX, "Lib", "site-packages") > > elif os.name == "mac": > if plat_specific: > if standard_lib: > return os.path.join(prefix, "Lib", "lib-dynload") > else: > return os.path.join(prefix, "Lib", "site-packages") > else: > if standard_lib: > return os.path.join(prefix, "Lib") > else: > return os.path.join(prefix, "Lib", "site-packages") > > elif os.name == "os2": > if standard_lib: > return os.path.join(PREFIX, "Lib") > else: > return os.path.join(PREFIX, "Lib", "site-packages") > > else: > raise DistutilsPlatformError( > "I don't know where Python installs its library " > "on platform '%s'" % os.name) > > > Under the posix section, distutils assumes that Python is installed in > /usr/lib/python-X.X, where it's really in /usr/lib64/python-X.X. This is > clearly a distutils bug. Distutils should be looking under lib64 for > 64-bit x86 systems. > > All linux x86-64 distributions use lib64 instead of lib for 64 bit > libraries. It looks like this code needs to be cleverer. Patches are welcome :-) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 15 2005) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Apr 15 22:08:29 2005 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Apr 15 22:08:31 2005 Subject: [Distutils] bdist_rpm and bdist on x86-64 In-Reply-To: References: <425FFD7C.5060602@egenix.com> Message-ID: <42601F3D.9000703@egenix.com> Jeremy Sanders wrote: > ... > Under the posix section, distutils assumes that Python is installed in > /usr/lib/python-X.X, where it's really in /usr/lib64/python-X.X. This is > clearly a distutils bug. Distutils should be looking under lib64 for > 64-bit x86 systems. > > All linux x86-64 distributions use lib64 instead of lib for 64 bit > libraries. Just as note: distutils should read this from the Makefile rather than trying to guess - whether or not lib64 is used depends on the distribution, the way the particular Python version was installed, etc. (e.g. a user installed Python version will still install to /usr/lib per default - even on 64-bit platforms). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 15 2005) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From jeremy at jeremysanders.net Fri Apr 15 22:26:36 2005 From: jeremy at jeremysanders.net (Jeremy Sanders) Date: Fri Apr 15 22:26:44 2005 Subject: [Distutils] bdist_rpm and bdist on x86-64 In-Reply-To: <42601F3D.9000703@egenix.com> References: <425FFD7C.5060602@egenix.com> <42601F3D.9000703@egenix.com> Message-ID: On Fri, 15 Apr 2005, M.-A. Lemburg wrote: > Just as note: distutils should read this from the Makefile rather > than trying to guess - whether or not lib64 is used depends on the > distribution, the way the particular Python version was installed, > etc. (e.g. a user installed Python version will still install > to /usr/lib per default - even on 64-bit platforms). Is this a packaging bug in fedora? Jeremy -- Jeremy Sanders http://www.jeremysanders.net/ Cambridge, UK Public Key Server PGP Key ID: E1AAE053 From mal at egenix.com Fri Apr 15 23:23:56 2005 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Apr 15 23:23:58 2005 Subject: [Distutils] bdist_rpm and bdist on x86-64 In-Reply-To: References: <425FFD7C.5060602@egenix.com> <42601F3D.9000703@egenix.com> Message-ID: <426030EC.3070209@egenix.com> Jeremy Sanders wrote: > On Fri, 15 Apr 2005, M.-A. Lemburg wrote: > >> Just as note: distutils should read this from the Makefile rather >> than trying to guess - whether or not lib64 is used depends on the >> distribution, the way the particular Python version was installed, >> etc. (e.g. a user installed Python version will still install >> to /usr/lib per default - even on 64-bit platforms). Should have been "/usr/local/lib/python2.4". > Is this a packaging bug in fedora? No, it's just the way most Linux distro's tweak the Python setup to support 64-bit along side with 32-bit ones (which then usually live in /usr/lib et al). You often have problems with this choice if you want to build things from source on x86-64. In most cases, you need to set LDFLAGS to -L/usr/lib64 or edit Makefiles. to adjust the compiler settings. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 15 2005) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From azubrow at galton.uchicago.edu Sat Apr 16 02:07:58 2005 From: azubrow at galton.uchicago.edu (Alexis Zubrow) Date: Sat Apr 16 02:08:05 2005 Subject: [Distutils] distutils and deb packages Message-ID: Hi. I was googling some things about distutils and debian packages, and came across some old emails that suggested that there was interest in adding deb packages as possible built distribution output. Do you know if anyone has actually done this? My version of distutils, 1.0.3, doesn't seem to have this functionality. Thanks for the info. Best, Alexis -- ------------------------------------------------------------ Alexis Zubrow CISES Phone: (773)834-9945 University of Chicago Fax: (773)834-1045 5734 S. Ellis Ave., rm 469 Chicago, IL 60637 ------------------------------------------------------------ From jeremy at jeremysanders.net Sat Apr 16 14:20:10 2005 From: jeremy at jeremysanders.net (Jeremy Sanders) Date: Sat Apr 16 14:20:17 2005 Subject: [Distutils] bdist_rpm and bdist on x86-64 In-Reply-To: <42601EBB.6010009@egenix.com> References: <425FFD7C.5060602@egenix.com> <42601EBB.6010009@egenix.com> Message-ID: On Fri, 15 Apr 2005, M.-A. Lemburg wrote: >> All linux x86-64 distributions use lib64 instead of lib for 64 bit >> libraries. It looks like this code needs to be cleverer. > > Patches are welcome :-) Attached is a patch which allows distutils to find the lib directory on x86-64. It uses sys.path to search for the Makefile. It also changes command/install.py to change platlib to lib64 if lib64 was returned by get_python_lib. It might be a bit messy for distutils of course, but it shouldn't affect non-x86-64 platforms. Opinions? Jeremy -- Jeremy Sanders http://www.jeremysanders.net/ Cambridge, UK Public Key Server PGP Key ID: E1AAE053 -------------- next part -------------- --- sysconfig.py~ 2005-03-03 11:08:02.000000000 +0000 +++ sysconfig.py 2005-04-16 12:20:03.499916704 +0100 @@ -99,8 +99,17 @@ prefix = plat_specific and EXEC_PREFIX or PREFIX if os.name == "posix": - libpython = os.path.join(prefix, - "lib", "python" + get_python_version()) + # search for Makefile in path to locate lib directory, in case + # python isn't in /prefix/lib (e.g. /prefix/lib64) + for i in sys.path: + if os.access( os.path.join(i, "config", "Makefile"), os.F_OK ): + libpython = i + break + else: + # Makefile not found in path + libpython = os.path.join(prefix, + "lib", "python" + get_python_version()) + if standard_lib: return libpython else: --- command/install.py~ 2005-01-20 19:14:17.000000000 +0000 +++ command/install.py 2005-04-16 13:12:07.264032376 +0100 @@ -10,6 +10,7 @@ import sys, os, string from types import * +from distutils.sysconfig import get_python_lib from distutils.core import Command from distutils.debug import DEBUG from distutils.sysconfig import get_config_vars @@ -36,17 +37,25 @@ 'data' : '$base', } + +# Normally unix distributions use lib to store python platlib +# However, under x86-64, platlib uses lib64 +unix_platlib = 'lib' +if get_python_lib().find('lib64') != -1: + unix_platlib = 'lib64' + INSTALL_SCHEMES = { 'unix_prefix': { 'purelib': '$base/lib/python$py_version_short/site-packages', - 'platlib': '$platbase/lib/python$py_version_short/site-packages', + 'platlib': + '$platbase/%s/python$py_version_short/site-packages' % unix_platlib, 'headers': '$base/include/python$py_version_short/$dist_name', 'scripts': '$base/bin', 'data' : '$base', }, 'unix_home': { 'purelib': '$base/lib/python', - 'platlib': '$base/lib/python', + 'platlib': '$base/%s/python' % unix_platlib, 'headers': '$base/include/python/$dist_name', 'scripts': '$base/bin', 'data' : '$base', From mss at mawhrin.net Sat Apr 16 22:14:37 2005 From: mss at mawhrin.net (Mikhail Sobolev) Date: Sat Apr 16 22:14:42 2005 Subject: [Distutils] distutils and deb packages In-Reply-To: References: Message-ID: <20050416201437.GB2272@mawhrin.net> On Fri, Apr 15, 2005 at 07:07:58PM -0500, Alexis Zubrow wrote: > Hi. I was googling some things about distutils and debian packages, and > came across some old emails that suggested that there was interest in > adding deb packages as possible built distribution output. Do you know > if anyone has actually done this? My version of distutils, 1.0.3, > doesn't seem to have this functionality. I believe the code is available in sandbox. I'm using it time to time to "debianize" python applications that are kind enough to provide setup.py file. Regards -- Misha -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://mail.python.org/pipermail/distutils-sig/attachments/20050417/237d47c6/attachment.pgp From pje at telecommunity.com Sun Apr 17 01:14:06 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun Apr 17 01:10:38 2005 Subject: [Distutils] Re: Eggs Meta-Data In-Reply-To: <426159ED.5070308@pythonweb.org> Message-ID: <5.1.1.6.0.20050416190310.02943080@mail.telecommunity.com> At 07:31 PM 4/16/05 +0100, James Gardner wrote: >Hi Phillip, > >I've got a couple of questions about Eggs, if I should be addressing it to >a mailing list rather than you could you please let me know? I imagine that the Distutils-SIG is probably a good place for discussion, so I'm cc:ing my response there. >I think Eggs is very much needed and will be extremely useful. I'd like to >start organising my packages in a sensible structure for eggs. The >package.eggs-info directory currently doesn't recursively include >directories within eggs-info, just single files. Is this deliberate or an >unfinished part of the code? It's deliberate; nobody has presented a compelling use case for putting anything other than a handful of files in the metadata directory. >Is eggs-info designed to contain documentation, scripts and examples as well. No; these would generally go in the package directories, where they are then retrievable via the pkg_resources module. The metadata is primarily for things that apply to the distribution as a whole, and that are needed by applications or servers to identify e.g. the role or contents of the distribution. > If so is there a convention whereby scripts can be copied to the Scripts > directory of the Python installation so that they can be easily executed > if an egg is actually installed? If they're included as top-level modules, then "python -m" is sufficient to invoke them if the .egg is on sys.path or PYTHONPATH. There is no "installation" of eggs, which is at least part of their charm. :) >Couldn't a similar thing happen with documentation so a user could have >one central place for documentation? Wouldn't this be useful? It might be useful to define some kind of metadata file to identify how to find documentation files within the included packages, but I don't really see the point of putting the documentation itself in the metadata directory, aside from perhaps release notes and the like. Note that eggs are primarily intended to be an *execution* format, not an installation tool or packaging system. So really the metadata is intended for programs to read and use, not for humans. Applications that need plugins often have some kind of configuration or deployment file that specifies how the accompanying code is to be used by the application, and that's what the metadata directory is for. >P.S. If you want to recursively include all files in eggs-info add >copy_tree to line 9 of setuptools\command\bdist_egg.py > >from distutils.dir_util import create_tree, remove_tree, ensure_relative, >mkpath, copy_tree > >and change line 190 and surrounding lines to: > > if self.egg_info: > > copy_tree(self.egg_info, egg_info) Actually, you've now convinced me that I shouldn't do this, so that people aren't tempted to cram all sorts of crazy things in there. ;) From james at pythonweb.org Sun Apr 17 03:08:22 2005 From: james at pythonweb.org (James Gardner) Date: Sun Apr 17 03:07:49 2005 Subject: [Distutils] Re: Eggs Meta-Data In-Reply-To: <5.1.1.6.0.20050416190310.02943080@mail.telecommunity.com> References: <5.1.1.6.0.20050416190310.02943080@mail.telecommunity.com> Message-ID: <4261B706.8090806@pythonweb.org> Phillip J. Eby wrote: > Note that eggs are primarily intended to be an *execution* format, not > an installation tool or packaging system. So really the metadata is > intended for programs to read and use, not for humans. Applications > that need plugins often have some kind of configuration or deployment > file that specifies how the accompanying code is to be used by the > application, and that's what the metadata directory is for. I hope I'm not treading on old ground here, but why not use the eggs format as a packaging system as well as an execution format? PEP 262 mentions that a package database should be able to answer the following questions: * Is distribution X on a system? * What version of distribution X is installed? * Where can the new version of distribution X be found? (This can be defined as either "a home page where the user can go and find a download link", or "a place where a program can find the newest version?" Both should probably be supported.) * What files did distribution X put on my system? * What distribution did the file x/y/z.py come from? * Has anyone modified x/y/z.py locally? * What other distributions does this software need? * What Python modules does this distribution provide? A directory of .egg files could serve as a database to provide answers to all these questions. You wouldn't need to install the eggs, just have them all on the PYTHONPATH. The questions about file x/y/z.py become irrelevant if x/y/z can't be modified. As far as I understand it in order to turn eggs into a really useful packaging system too, all you would need to do is write an install program to download eggs automatically (from the required dependency URLs listed or a central server) into a particular directory and have them automatically added to PYTHONPATH. Eggs would be easy to uninstall.. just delete them. Different egg versions have different names so you could install multiple versions of the same package. The installer could then run the test suite if one existed and autogenerate documentation to a specified location. Surely it would be quite easy to use eggs as a basis for something similar to ruby gems but without the necessity of actually extracting the files. Is there any reason why eggs aren't being thought of in this way? Is there any drawback in extending the scope to use them as a packaging system too? Thanks, James From bob at redivi.com Sun Apr 17 03:48:02 2005 From: bob at redivi.com (Bob Ippolito) Date: Sun Apr 17 03:48:10 2005 Subject: [Distutils] Re: Eggs Meta-Data In-Reply-To: <4261B706.8090806@pythonweb.org> References: <5.1.1.6.0.20050416190310.02943080@mail.telecommunity.com> <4261B706.8090806@pythonweb.org> Message-ID: <8285de94c36fe749563177e01bb84d46@redivi.com> On Apr 16, 2005, at 9:08 PM, James Gardner wrote: > Phillip J. Eby wrote: > >> Note that eggs are primarily intended to be an *execution* format, >> not an installation tool or packaging system. So really the metadata >> is intended for programs to read and use, not for humans. >> Applications that need plugins often have some kind of configuration >> or deployment file that specifies how the accompanying code is to be >> used by the application, and that's what the metadata directory is >> for. > > I hope I'm not treading on old ground here, but why not use the eggs > format as a packaging system as well as an execution format? Why not solve every other problem Python has in one fell swoop? :) -bob From pje at telecommunity.com Sun Apr 17 05:22:44 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun Apr 17 05:19:11 2005 Subject: [Distutils] Re: Eggs Meta-Data In-Reply-To: <4261B706.8090806@pythonweb.org> References: <5.1.1.6.0.20050416190310.02943080@mail.telecommunity.com> <5.1.1.6.0.20050416190310.02943080@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20050416231348.0391ca40@mail.telecommunity.com> At 02:08 AM 4/17/05 +0100, James Gardner wrote: >Phillip J. Eby wrote: > >>Note that eggs are primarily intended to be an *execution* format, not an >>installation tool or packaging system. So really the metadata is >>intended for programs to read and use, not for humans. Applications that >>need plugins often have some kind of configuration or deployment file >>that specifies how the accompanying code is to be used by the >>application, and that's what the metadata directory is for. > >I hope I'm not treading on old ground here, but why not use the eggs >format as a packaging system as well as an execution format? Because I have no interest in trying to compete with existing packaging systems; it's not a wise investment of time. There are no existing execution formats for Python, however, that do what eggs can, so it's a more reasonable investment to compete in that sphere. >As far as I understand it in order to turn eggs into a really useful >packaging system too, all you would need to do is write an install program >to download eggs automatically (from the required dependency URLs listed >or a central server) into a particular directory and have them >automatically added to PYTHONPATH. Yep. I'm just not planning to write that myself at the moment. But pkg_resources will have some functions you can give a callback to, to let you know about missing dependencies so you can invoke a downloader. The thing is, different applications will have different policies and UIs for downloading and even configuring where to download from. So, the core system isn't going to do any downloading. It will just search where it's told to search, and add things to sys.path that you ask for, and call you back when things needed to fulfill your requests are missing. >Eggs would be easy to uninstall.. just delete them. Different egg versions >have different names so you could install multiple versions of the same >package. The installer could then run the test suite if one existed and >autogenerate documentation to a specified location. Surely it would be >quite easy to use eggs as a basis for something similar to ruby gems but >without the necessity of actually extracting the files. > >Is there any reason why eggs aren't being thought of in this way? Is there >any drawback in extending the scope to use them as a packaging system too? Yes, I'll have more code to write then. ;) More to the point, once you step outside of the "execution format" realm, you start to get into massive disputes about what policies should exist and I really don't want to spend a lot of time arguing about that stuff. Let those who want particular policies write their own bloody installers, documentation generators, and autotesters. :) More to the point, when we have a solid execution format, it'll be easier to think about those things than it is now. From james at pythonweb.org Sun Apr 17 13:24:29 2005 From: james at pythonweb.org (James Gardner) Date: Sun Apr 17 13:23:57 2005 Subject: [Distutils] Re: Eggs Meta-Data In-Reply-To: <5.1.1.6.0.20050416231348.0391ca40@mail.telecommunity.com> References: <5.1.1.6.0.20050416190310.02943080@mail.telecommunity.com> <5.1.1.6.0.20050416190310.02943080@mail.telecommunity.com> <5.1.1.6.0.20050416231348.0391ca40@mail.telecommunity.com> Message-ID: <4262476D.3070202@pythonweb.org> Phillip J. Eby wrote: > Yes, I'll have more code to write then. ;) More to the point, once > you step outside of the "execution format" realm, you start to get > into massive disputes about what policies should exist and I really > don't want to spend a lot of time arguing about that stuff. Fair points, I hope I haven't implied that Eggs isn't good enough as just an execution format, I said right at the start of my first email that I think Eggs is very much needed and will be extremely useful and I'm quite excited about using it for my software. > More to the point, when we have a solid execution format, it'll be > easier to think about those things than it is now. That's true too. > Let those who want particular policies write their own bloody > installers, documentation generators, and autotesters. :) I'm still tempted to have a go at writing a package manager around eggs for my software so might just give it a go for interest.. eggs have so many useful features.. I agree though that as a serious solution you would need to arbitrate a serious amount of discussion to please everyone. Thanks for your answers anyway, much appreciated. James From mal at egenix.com Mon Apr 18 20:05:15 2005 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Apr 18 20:05:19 2005 Subject: [Distutils] bdist_rpm and bdist on x86-64 In-Reply-To: References: <425FFD7C.5060602@egenix.com> <42601EBB.6010009@egenix.com> Message-ID: <4263F6DB.9040603@egenix.com> Jeremy Sanders wrote: > On Fri, 15 Apr 2005, M.-A. Lemburg wrote: > >>> All linux x86-64 distributions use lib64 instead of lib for 64 bit >>> libraries. It looks like this code needs to be cleverer. >> >> >> Patches are welcome :-) > > > Attached is a patch which allows distutils to find the lib directory on > x86-64. It uses sys.path to search for the Makefile. > > It also changes command/install.py to change platlib to lib64 if lib64 > was returned by get_python_lib. > > It might be a bit messy for distutils of course, but it shouldn't affect > non-x86-64 platforms. > > Opinions? > > Jeremy > > > ------------------------------------------------------------------------ > > --- sysconfig.py~ 2005-03-03 11:08:02.000000000 +0000 > +++ sysconfig.py 2005-04-16 12:20:03.499916704 +0100 > @@ -99,8 +99,17 @@ > prefix = plat_specific and EXEC_PREFIX or PREFIX > > if os.name == "posix": > - libpython = os.path.join(prefix, > - "lib", "python" + get_python_version()) > + # search for Makefile in path to locate lib directory, in case > + # python isn't in /prefix/lib (e.g. /prefix/lib64) > + for i in sys.path: > + if os.access( os.path.join(i, "config", "Makefile"), os.F_OK ): Why not os.exists() ? > + libpython = i > + break > + else: > + # Makefile not found in path > + libpython = os.path.join(prefix, > + "lib", "python" + get_python_version()) > + Wouldn't checking for the lib file in either lib64 or lib be more reliable ? > if standard_lib: > return libpython > else: > --- command/install.py~ 2005-01-20 19:14:17.000000000 +0000 > +++ command/install.py 2005-04-16 13:12:07.264032376 +0100 > @@ -10,6 +10,7 @@ > > import sys, os, string > from types import * > +from distutils.sysconfig import get_python_lib > from distutils.core import Command > from distutils.debug import DEBUG > from distutils.sysconfig import get_config_vars > @@ -36,17 +37,25 @@ > 'data' : '$base', > } > > + > +# Normally unix distributions use lib to store python platlib > +# However, under x86-64, platlib uses lib64 > +unix_platlib = 'lib' > +if get_python_lib().find('lib64') != -1: > + unix_platlib = 'lib64' > + Hmm, this is not entirely correct, e.g. Suse 9.2 puts the site-packages directory and all the other .py files into /usr/lib64 as well - not only the platforma dependent files. Not sure what other AMD64 distros do... but I have a feeling the /lib/ should *always* be replaced by unix_platlib. > INSTALL_SCHEMES = { > 'unix_prefix': { > 'purelib': '$base/lib/python$py_version_short/site-packages', > - 'platlib': '$platbase/lib/python$py_version_short/site-packages', > + 'platlib': > + '$platbase/%s/python$py_version_short/site-packages' % unix_platlib, > 'headers': '$base/include/python$py_version_short/$dist_name', > 'scripts': '$base/bin', > 'data' : '$base', > }, > 'unix_home': { > 'purelib': '$base/lib/python', > - 'platlib': '$base/lib/python', > + 'platlib': '$base/%s/python' % unix_platlib, > 'headers': '$base/include/python/$dist_name', > 'scripts': '$base/bin', > 'data' : '$base', > > > ------------------------------------------------------------------------ > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG@python.org > http://mail.python.org/mailman/listinfo/distutils-sig -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 18 2005) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From jeremy at jeremysanders.net Mon Apr 18 20:30:36 2005 From: jeremy at jeremysanders.net (Jeremy Sanders) Date: Mon Apr 18 20:30:46 2005 Subject: [Distutils] bdist_rpm and bdist on x86-64 In-Reply-To: <4263F6DB.9040603@egenix.com> References: <425FFD7C.5060602@egenix.com> <42601EBB.6010009@egenix.com> <4263F6DB.9040603@egenix.com> Message-ID: On Mon, 18 Apr 2005, M.-A. Lemburg wrote: > Why not os.exists() ? Probably better :-) > Wouldn't checking for the lib file in either lib64 > or lib be more reliable ? That might be better. I don't know whether it's possible that more than one python is installed (one in /lib and one in /lib64). > Hmm, this is not entirely correct, e.g. Suse 9.2 puts > the site-packages directory and all the other .py files > into /usr/lib64 as well - not only the platforma dependent > files. > > Not sure what other AMD64 distros do... but I have a feeling > the /lib/ should *always* be replaced by unix_platlib. It looks like RedHat/Fedora patch their package to only put the platform specific files in /lib64 (that's how I made my patch). Perhaps this isn't a good idea to do then :-( I wonder whether it would be possible for distribution to set these values somewhere. Couldn't python have a sys function to return its Makefile? (anyway, I've managed to get my setup.py to work with the built-in distutils...) Jeremy -- Jeremy Sanders http://www.jeremysanders.net/ Cambridge, UK Public Key Server PGP Key ID: E1AAE053 From mal at egenix.com Tue Apr 19 15:00:49 2005 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Apr 19 15:00:50 2005 Subject: [Distutils] bdist_rpm and bdist on x86-64 In-Reply-To: References: <4263F6DB.9040603@egenix.com> Message-ID: <42650101.3080903@egenix.com> Jeremy Sanders wrote: > On Mon, 18 Apr 2005, M.-A. Lemburg wrote: > >> Why not os.exists() ? s/os.exists/os.path.exists > Probably better :-) > >> Wouldn't checking for the lib file in either lib64 >> or lib be more reliable ? > > That might be better. I don't know whether it's possible that more than > one python is installed (one in /lib and one in /lib64). That would certainly be possible. >> Hmm, this is not entirely correct, e.g. Suse 9.2 puts >> the site-packages directory and all the other .py files >> into /usr/lib64 as well - not only the platforma dependent >> files. >> >> Not sure what other AMD64 distros do... but I have a feeling >> the /lib/ should *always* be replaced by unix_platlib. > > > It looks like RedHat/Fedora patch their package to only put the platform > specific files in /lib64 (that's how I made my patch). > > Perhaps this isn't a good idea to do then :-( I wonder whether it would > be possible for distribution to set these values somewhere. Couldn't > python have a sys function to return its Makefile? It's rather easy to find the right Makefile: just look for os.py (the landmark used in Python to initialize sys.path on startup), join it's path with "config" and you're there. More details can be found in Modules/getpath.c of the Python source distribution. You can then have distutils parse that file to extract the other details. Note that this doesn't work if you're importing from a ZIP file. > (anyway, I've managed to get my setup.py to work with the built-in > distutils...) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 19 2005) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From misa at redhat.com Tue Apr 19 15:51:37 2005 From: misa at redhat.com (Mihai Ibanescu) Date: Tue Apr 19 15:51:44 2005 Subject: [Distutils] bdist_rpm and bdist on x86-64 In-Reply-To: <42650101.3080903@egenix.com> References: <4263F6DB.9040603@egenix.com> <42650101.3080903@egenix.com> Message-ID: <20050419135137.GH9562@abulafia.devel.redhat.com> On Tue, Apr 19, 2005 at 03:00:49PM +0200, M.-A. Lemburg wrote: > Jeremy Sanders wrote: > > On Mon, 18 Apr 2005, M.-A. Lemburg wrote: > > > >> Why not os.exists() ? > > s/os.exists/os.path.exists > > > Probably better :-) > > > >> Wouldn't checking for the lib file in either lib64 > >> or lib be more reliable ? > > > > That might be better. I don't know whether it's possible that more than > > one python is installed (one in /lib and one in /lib64). > > That would certainly be possible. > > >> Hmm, this is not entirely correct, e.g. Suse 9.2 puts > >> the site-packages directory and all the other .py files > >> into /usr/lib64 as well - not only the platforma dependent > >> files. > >> > >> Not sure what other AMD64 distros do... but I have a feeling > >> the /lib/ should *always* be replaced by unix_platlib. > > > > > > It looks like RedHat/Fedora patch their package to only put the platform > > specific files in /lib64 (that's how I made my patch). > > > > Perhaps this isn't a good idea to do then :-( I wonder whether it would > > be possible for distribution to set these values somewhere. Couldn't > > python have a sys function to return its Makefile? Fedora does it this way because of .noarch.rpm packages. Pure python libraries should run just fine both on x86 and x86_64, and since it's /usr/lib on x86, x86_64 has to know about /usr/lib too (which is sort of confusing). I believe you can get away without patching anything, if you do: from distutils import sysconfig print sysconfig.get_python_lib() to which you either pass plat_specific = 0 or 1. This will properly parse the right Makefile (which is probably what you ended up doing). Misa From jeremy at jeremysanders.net Tue Apr 19 15:55:03 2005 From: jeremy at jeremysanders.net (Jeremy Sanders) Date: Tue Apr 19 15:55:13 2005 Subject: [Distutils] bdist_rpm and bdist on x86-64 In-Reply-To: <20050419135137.GH9562@abulafia.devel.redhat.com> References: <4263F6DB.9040603@egenix.com> <42650101.3080903@egenix.com> <20050419135137.GH9562@abulafia.devel.redhat.com> Message-ID: On Tue, 19 Apr 2005, Mihai Ibanescu wrote: > I believe you can get away without patching anything, if you do: > > from distutils import sysconfig > print sysconfig.get_python_lib() > > to which you either pass plat_specific = 0 or 1. This will properly > parse the right Makefile (which is probably what you ended up doing). The problem is that this doesn't find the Makefile if you use the unpatched distutils on Fedora on x86_64. Fedora patches distutils to look in lib64. The unpatched distutils looks in lib, and never finds it. It would be nice if Python knew where its Makefile was. Jeremy -- Jeremy Sanders http://www.jeremysanders.net/ Cambridge, UK Public Key Server PGP Key ID: E1AAE053 From misa at redhat.com Tue Apr 19 19:45:10 2005 From: misa at redhat.com (Mihai Ibanescu) Date: Tue Apr 19 19:45:15 2005 Subject: [Distutils] bdist_rpm and bdist on x86-64 In-Reply-To: References: <4263F6DB.9040603@egenix.com> <42650101.3080903@egenix.com> <20050419135137.GH9562@abulafia.devel.redhat.com> Message-ID: <20050419174510.GL9562@abulafia.devel.redhat.com> On Tue, Apr 19, 2005 at 02:55:03PM +0100, Jeremy Sanders wrote: > On Tue, 19 Apr 2005, Mihai Ibanescu wrote: > > >I believe you can get away without patching anything, if you do: > > > >from distutils import sysconfig > >print sysconfig.get_python_lib() > > > >to which you either pass plat_specific = 0 or 1. This will properly > >parse the right Makefile (which is probably what you ended up doing). > > The problem is that this doesn't find the Makefile if you use the > unpatched distutils on Fedora on x86_64. Fedora patches distutils to look > in lib64. The unpatched distutils looks in lib, and never finds it. Sorry if I missed it from previous e-mails. Why don't you use Fedora's patched distutils then? :-) > It would be nice if Python knew where its Makefile was. Python does, if it's in the standard location. /usr/lib64 was not necessarily something that worked well for Python, at least not from the very beginning, and that's because there is a prefix and an exec_prefix, and then an unpatched python will just slap a lib after that. If multilib were developed to use a totally different prefix than /usr (like /usr64), it would definitely have been easier. That, or I am missing the obvious fixes :-) Misa From cludwig at cdc.informatik.tu-darmstadt.de Wed Apr 20 10:19:14 2005 From: cludwig at cdc.informatik.tu-darmstadt.de (Christoph Ludwig) Date: Wed Apr 20 10:19:17 2005 Subject: [Distutils] [build_ext] more than one external library In-Reply-To: <20050415130926.GC2724@lap200.cdc.informatik.tu-darmstadt.de> References: <20050415130926.GC2724@lap200.cdc.informatik.tu-darmstadt.de> Message-ID: <20050420081914.GD2730@lap200.cdc.informatik.tu-darmstadt.de> Hi, On Fri, Apr 15, 2005 at 03:09:26PM +0200, Christoph Ludwig wrote: > I use the distutils shipped with Python 2.4 on a i686-pc-linux-gnu system. > > I am wondering how can I pass more than one external library to distutils' > build_ext option (on the command line or even better via setup.cfg)? can anyone help with this one? In case you wonder why I need to pass the library names on the command line of setup.py: My extension uses several Boost libraries. Boost uses some kind of "library name mangling" to encode the compiler, library version and build options (single vs. multi threaded, release vs. debug build) in the library names. I cannot know which build of the Boost library the user of my extension will want / need to link against. (E.g., if he user's python interpreter was built without thread support then the extension must not be linked against libboost_*mt*.) So I need to offer the users of my extension a way to override the default choice of external libraries. > In analogy to --include-dirs I'd expect > > ./setup.py build_ext --libraries A:B > > to generate a linker call similar to: > > g++ -lA -lB # ... other arguments and options > > But I get: > > g++ -lA:B # ... > > I also tried > > ./setup.py build_ext --libraries 'A B' > ./setup.py build_ext --libraries A B > ./setup.py build_ext --libraries A,B > ./setup.py build_ext --libraries A --libraries B > > but to no avail. (In the last case the second --libraries option seems to > overwrite the first one.) > > Is this feature missing and I need to put the library list in the libraries > option of the respective Extension constructor in setup.py? Or did I fail to > try the correct syntax? Regards Christoph -- http://www.informatik.tu-darmstadt.de/TI/Mitarbeiter/cludwig.html LiDIA: http://www.informatik.tu-darmstadt.de/TI/LiDIA/Welcome.html From mal at egenix.com Wed Apr 20 10:39:24 2005 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Apr 20 10:39:27 2005 Subject: [Distutils] bdist_rpm and bdist on x86-64 In-Reply-To: <20050419174510.GL9562@abulafia.devel.redhat.com> References: <4263F6DB.9040603@egenix.com> <42650101.3080903@egenix.com> <20050419135137.GH9562@abulafia.devel.redhat.com> <20050419174510.GL9562@abulafia.devel.redhat.com> Message-ID: <4266153C.1000901@egenix.com> Mihai Ibanescu wrote: > On Tue, Apr 19, 2005 at 02:55:03PM +0100, Jeremy Sanders wrote: > >>On Tue, 19 Apr 2005, Mihai Ibanescu wrote: >> >> >>>I believe you can get away without patching anything, if you do: >>> >> >>>from distutils import sysconfig >> >>>print sysconfig.get_python_lib() >>> >>>to which you either pass plat_specific = 0 or 1. This will properly >>>parse the right Makefile (which is probably what you ended up doing). >> >>The problem is that this doesn't find the Makefile if you use the >>unpatched distutils on Fedora on x86_64. Fedora patches distutils to look >>in lib64. The unpatched distutils looks in lib, and never finds it. > > > Sorry if I missed it from previous e-mails. Why don't you use Fedora's patched > distutils then? :-) I wonder why Linux distros always have to make things more complicated by patching code in the standard lib or using non-standard compile time options... >>It would be nice if Python knew where its Makefile was. Jeremy, didn't you get my last posting ? It is rather easy to find the Makefile since the config/ dir is stored in the same place as the rest of the Python std lib (provided the distros didn't munge that fact as well, like e.g. some which remove distutils from the std lib ). > Python does, if it's in the standard location. /usr/lib64 was not necessarily > something that worked well for Python, at least not from the very beginning, > and that's because there is a prefix and an exec_prefix, and then an unpatched > python will just slap a lib after that. If multilib were developed to use a > totally different prefix than /usr (like /usr64), it would definitely have > been easier. > > That, or I am missing the obvious fixes :-) I think that Linux distros should approach the Python developers first before fiddling around with Python internals. We would come up with a solution that works for everybody, not just distro XYZ users. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 20 2005) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From jeremy at jeremysanders.net Wed Apr 20 10:49:08 2005 From: jeremy at jeremysanders.net (Jeremy Sanders) Date: Wed Apr 20 10:49:29 2005 Subject: [Distutils] bdist_rpm and bdist on x86-64 In-Reply-To: <4266153C.1000901@egenix.com> References: <4263F6DB.9040603@egenix.com> <42650101.3080903@egenix.com> <20050419135137.GH9562@abulafia.devel.redhat.com> <20050419174510.GL9562@abulafia.devel.redhat.com> <4266153C.1000901@egenix.com> Message-ID: On Wed, 20 Apr 2005, M.-A. Lemburg wrote: > Jeremy, didn't you get my last posting ? It is rather easy > to find the Makefile since the config/ dir is stored in the > same place as the rest of the Python std lib (provided the > distros didn't munge that fact as well, like e.g. some > which remove distutils from the std lib ). Yes, we could just do something like os.path.join(os.path.dirname(os.__file__), 'config', 'Makefile') to find the Makefile location (as long as os doesn't become built-in, like sys, or ends up in a zip file). There's still the problem of working out where x86-64 non specfic things should get installed in lib64 or lib. I don't see an easy answer there. It is a real shame that RedHat/Fedora didn't get this sorted before they started distributing python on x86-64. I wonder what SuSE are doing with lib/lib64? Jeremy -- Jeremy Sanders http://www.jeremysanders.net/ Cambridge, UK Public Key Server PGP Key ID: E1AAE053 From mal at egenix.com Wed Apr 20 11:00:14 2005 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Apr 20 11:00:18 2005 Subject: [Distutils] bdist_rpm and bdist on x86-64 In-Reply-To: <20050419135137.GH9562@abulafia.devel.redhat.com> References: <4263F6DB.9040603@egenix.com> <42650101.3080903@egenix.com> <20050419135137.GH9562@abulafia.devel.redhat.com> Message-ID: <42661A1E.5070601@egenix.com> Mihai Ibanescu wrote: >>>It looks like RedHat/Fedora patch their package to only put the platform >>>specific files in /lib64 (that's how I made my patch). >>> >>>Perhaps this isn't a good idea to do then :-( I wonder whether it would >>>be possible for distribution to set these values somewhere. Couldn't >>>python have a sys function to return its Makefile? > > > Fedora does it this way because of .noarch.rpm packages. Pure python > libraries should run just fine both on x86 and x86_64, and since it's /usr/lib > on x86, x86_64 has to know about /usr/lib too (which is sort of confusing). Not only that, I doubt that this will work well unless you now have two site-packages dirs (one in /usr/lib for Python only extensions and one in /usr/lib64 for combined Python and binary extensions). > I believe you can get away without patching anything, if you do: > > from distutils import sysconfig > print sysconfig.get_python_lib() > > to which you either pass plat_specific = 0 or 1. This will properly parse the > right Makefile (which is probably what you ended up doing). Uhm, this is the function that Jeremy was patching - the problem being that the "lib" part of the path is hard-coded into this function. It doesn't parse the Makefile, BTW, and plat_specific only adds site-packages to the returned directory - it doesn't have anything to do with platform specific code. The Makefile is the only place where Python itself can look up how it was configured. It's easy to find: distutils.sysconfig.get_makefile_filename(); distutils.sysconfig.parse_makefile(filename) then does the rest. Or in one go: distutils.sysconfig.get_config_vars(). The interesting parts of the Makefile are these variables: # Detailed destination directories BINLIBDEST= $(LIBDIR)/python$(VERSION) LIBDEST= $(SCRIPTDIR)/python$(VERSION) INCLUDEPY= $(INCLUDEDIR)/python$(VERSION) CONFINCLUDEPY= $(CONFINCLUDEDIR)/python$(VERSION) LIBP= $(LIBDIR)/python$(VERSION) I guess we should change the install.py to use these values instead of building its own little world of pathnames. What do you think ? -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 20 2005) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Apr 20 11:07:37 2005 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Apr 20 11:07:39 2005 Subject: [Distutils] bdist_rpm and bdist on x86-64 In-Reply-To: References: <4263F6DB.9040603@egenix.com><42650101.3080903@egenix.com> <20050419135137.GH9562@abulafia.devel.redhat.com> <4266153C.1000901@egenix.com> Message-ID: <42661BD9.8020806@egenix.com> Jeremy Sanders wrote: > On Wed, 20 Apr 2005, M.-A. Lemburg wrote: > >> Jeremy, didn't you get my last posting ? It is rather easy >> to find the Makefile since the config/ dir is stored in the >> same place as the rest of the Python std lib (provided the >> distros didn't munge that fact as well, like e.g. some >> which remove distutils from the std lib ). > > > Yes, we could just do something like > > os.path.join(os.path.dirname(os.__file__), 'config', 'Makefile') > > to find the Makefile location (as long as os doesn't become built-in, > like sys, or ends up in a zip file). > > There's still the problem of working out where x86-64 non specfic things > should get installed in lib64 or lib. I don't see an easy answer there. > > It is a real shame that RedHat/Fedora didn't get this sorted before they > started distributing python on x86-64. I wonder what SuSE are doing with > lib/lib64? SuSE 9.2 puts the 64-bit version into /usr/lib64 (the complete lib, including the site-packages) and the 32-bit version into /usr/lib. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 20 2005) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From seefeld at sympatico.ca Wed Apr 20 14:38:11 2005 From: seefeld at sympatico.ca (Stefan Seefeld) Date: Wed Apr 20 14:39:31 2005 Subject: [Distutils] [build_ext] more than one external library In-Reply-To: <20050420081914.GD2730@lap200.cdc.informatik.tu-darmstadt.de> References: <20050415130926.GC2724@lap200.cdc.informatik.tu-darmstadt.de> <20050420081914.GD2730@lap200.cdc.informatik.tu-darmstadt.de> Message-ID: <42664D33.90706@sympatico.ca> Christoph Ludwig wrote: > In case you wonder why I need to pass the library names on the command line > of setup.py: My extension uses several Boost libraries. Boost uses some kind > of "library name mangling" to encode the compiler, library version and build > options (single vs. multi threaded, release vs. debug build) in the library > names. I cannot know which build of the Boost library the user of my extension > will want / need to link against. (E.g., if he user's python interpreter was > built without thread support then the extension must not be linked against > libboost_*mt*.) I once had the same concern, but many others, such as build_ext being very inflexible in its way to compile extension modules. What I ended up with is a set of custom commands ('config', 'build_ext', 'test', 'install...') to allow subprojects to be built with standard autotools (autoconf, make), but being ultimately controlled by distutils. I then could take advantage of distutils packaging infrastructure ('sdist', 'bdist'). Doing this the build/install procedure is now python setup.py config python setup.py build python setup.py test (optional) python setup.py install It all works quite well, though I would prefer not to use a hybride build system. (I'm still waiting for alternatives such as scons or boost.build to mature.) HTH, Stefan From cludwig at cdc.informatik.tu-darmstadt.de Wed Apr 20 15:23:33 2005 From: cludwig at cdc.informatik.tu-darmstadt.de (Christoph Ludwig) Date: Wed Apr 20 15:23:35 2005 Subject: [Distutils] [build_ext] more than one external library In-Reply-To: <42664D33.90706@sympatico.ca> References: <20050415130926.GC2724@lap200.cdc.informatik.tu-darmstadt.de> <20050420081914.GD2730@lap200.cdc.informatik.tu-darmstadt.de> <42664D33.90706@sympatico.ca> Message-ID: <20050420132333.GE2730@lap200.cdc.informatik.tu-darmstadt.de> Hi, On Wed, Apr 20, 2005 at 08:38:11AM -0400, Stefan Seefeld wrote: > I once had the same concern, but many others, such as build_ext being very > inflexible in its way to compile extension modules. > > What I ended up with is a set of custom commands ('config', 'build_ext', > 'test', > 'install...') to allow subprojects to be built with standard autotools > (autoconf, > make), but being ultimately controlled by distutils. I then could take > advantage > of distutils packaging infrastructure ('sdist', 'bdist'). Hm, I didn't look into creating custom commands yet. It's certainly an interesting idea provided it's not too complex. (This is my first project where I use Python so I am still somewhat green.) > Doing this the build/install procedure is now > > python setup.py config > python setup.py build > python setup.py test (optional) > python setup.py install In the short term I will probably settle for a README that tells the user to modify the library names in setup.py. But in the long term your approach would allow me to merge the Python extension module with the library it calls. Are you in a position (and willing, of course) to share the code for above mentioned commands? > It all works quite well, though I would prefer not to use a hybride build > system. > (I'm still waiting for alternatives such as scons or boost.build to mature.) Agreed. For the time being we will have to make do, though. Regards Christoph -- http://www.informatik.tu-darmstadt.de/TI/Mitarbeiter/cludwig.html LiDIA: http://www.informatik.tu-darmstadt.de/TI/LiDIA/Welcome.html From seefeld at sympatico.ca Wed Apr 20 15:37:13 2005 From: seefeld at sympatico.ca (Stefan Seefeld) Date: Wed Apr 20 15:38:31 2005 Subject: [Distutils] [build_ext] more than one external library In-Reply-To: <20050420132333.GE2730@lap200.cdc.informatik.tu-darmstadt.de> References: <20050415130926.GC2724@lap200.cdc.informatik.tu-darmstadt.de> <20050420081914.GD2730@lap200.cdc.informatik.tu-darmstadt.de> <42664D33.90706@sympatico.ca> <20050420132333.GE2730@lap200.cdc.informatik.tu-darmstadt.de> Message-ID: <42665B09.1020601@sympatico.ca> Christoph Ludwig wrote: >>python setup.py config >>python setup.py build >>python setup.py test (optional) >>python setup.py install > > > In the short term I will probably settle for a README that tells the user to > modify the library names in setup.py. But in the long term your approach would > allow me to merge the Python extension module with the library it calls. > > Are you in a position (and willing, of course) to share the code for above > mentioned commands? Sure. Have a look at http://synopsis.fresco.org/viewsvn/synopsis-Synopsis/trunk/Synopsis/dist/command/ * 'config' calls 'configure' on a number of subprojects * 'build_syn_ext' runs 'make' on extension modules (completely bypassing distutils' own 'Extension' data structure !) * 'build_syn_clib' runs 'make' on subprojects that generate standalone DSOs, not meant to be directly accessed as python extensions * 'test' runs a unit/regression test suite (qmtest based) Good luck ! Stefan From cludwig at cdc.informatik.tu-darmstadt.de Wed Apr 20 17:05:56 2005 From: cludwig at cdc.informatik.tu-darmstadt.de (Christoph Ludwig) Date: Wed Apr 20 17:05:59 2005 Subject: [Distutils] [build_ext] more than one external library In-Reply-To: <42665B09.1020601@sympatico.ca> References: <20050415130926.GC2724@lap200.cdc.informatik.tu-darmstadt.de> <20050420081914.GD2730@lap200.cdc.informatik.tu-darmstadt.de> <42664D33.90706@sympatico.ca> <20050420132333.GE2730@lap200.cdc.informatik.tu-darmstadt.de> <42665B09.1020601@sympatico.ca> Message-ID: <20050420150556.GF2730@lap200.cdc.informatik.tu-darmstadt.de> On Wed, Apr 20, 2005 at 09:37:13AM -0400, Stefan Seefeld wrote: > >Are you in a position (and willing, of course) to share the code for above > >mentioned commands? > > Sure. Have a look at > > http://synopsis.fresco.org/viewsvn/synopsis-Synopsis/trunk/Synopsis/dist/command/ > > * 'config' calls 'configure' on a number of subprojects > * 'build_syn_ext' runs 'make' on extension modules (completely bypassing > distutils' > own 'Extension' data structure !) > * 'build_syn_clib' runs 'make' on subprojects that generate standalone > DSOs, not > meant to be directly accessed as python extensions > * 'test' runs a unit/regression test suite (qmtest based) > > Good luck ! Thanks! I will try it as soon as I have some free time. Best regards Christoph -- http://www.informatik.tu-darmstadt.de/TI/Mitarbeiter/cludwig.html LiDIA: http://www.informatik.tu-darmstadt.de/TI/LiDIA/Welcome.html From misa at redhat.com Sat Apr 23 00:49:32 2005 From: misa at redhat.com (Mihai Ibanescu) Date: Sat Apr 23 00:49:24 2005 Subject: [Distutils] bdist_rpm and bdist on x86-64 In-Reply-To: <42661A1E.5070601@egenix.com> References: <4263F6DB.9040603@egenix.com> <42650101.3080903@egenix.com> <20050419135137.GH9562@abulafia.devel.redhat.com> <42661A1E.5070601@egenix.com> Message-ID: <20050422224932.GC24662@abulafia.devel.redhat.com> On Wed, Apr 20, 2005 at 11:00:14AM +0200, M.-A. Lemburg wrote: > Mihai Ibanescu wrote: > >>>It looks like RedHat/Fedora patch their package to only put the platform > >>>specific files in /lib64 (that's how I made my patch). > >>> > >>>Perhaps this isn't a good idea to do then :-( I wonder whether it would > >>>be possible for distribution to set these values somewhere. Couldn't > >>>python have a sys function to return its Makefile? > > > > > > Fedora does it this way because of .noarch.rpm packages. Pure python > > libraries should run just fine both on x86 and x86_64, and since it's /usr/lib > > on x86, x86_64 has to know about /usr/lib too (which is sort of confusing). > > Not only that, I doubt that this will work well unless > you now have two site-packages dirs (one in /usr/lib for > Python only extensions and one in /usr/lib64 for combined > Python and binary extensions). > > > I believe you can get away without patching anything, if you do: > > > > from distutils import sysconfig > > print sysconfig.get_python_lib() > > > > to which you either pass plat_specific = 0 or 1. This will properly parse the > > right Makefile (which is probably what you ended up doing). > > Uhm, this is the function that Jeremy was patching - the problem > being that the "lib" part of the path is hard-coded into this > function. > > It doesn't parse the Makefile, BTW, and plat_specific only > adds site-packages to the returned directory - it doesn't > have anything to do with platform specific code. Indeed it doesn't parse the Makefile. I believe I was remembering something else about Makefiles. Sorry. This is what we have in Fedora: if os.name == "posix": if plat_specific or standard_lib: lib = "lib64" else: lib = "lib" libpython = os.path.join(prefix, lib, "python" + get_python_version()) if standard_lib: return libpython else: return os.path.join(libpython, "site-packages") This is what stock python 2.4.1 does: libpython = os.path.join(prefix, "lib", "python" + get_python_version()) if standard_lib: return libpython else: return os.path.join(libpython, "site-packages") > The Makefile is the only place where Python itself can > look up how it was configured. It's easy to find: > distutils.sysconfig.get_makefile_filename(); > distutils.sysconfig.parse_makefile(filename) then does the > rest. Or in one go: distutils.sysconfig.get_config_vars(). > > The interesting parts of the Makefile are these variables: > > # Detailed destination directories > BINLIBDEST= $(LIBDIR)/python$(VERSION) > LIBDEST= $(SCRIPTDIR)/python$(VERSION) > INCLUDEPY= $(INCLUDEDIR)/python$(VERSION) > CONFINCLUDEPY= $(CONFINCLUDEDIR)/python$(VERSION) > LIBP= $(LIBDIR)/python$(VERSION) > > I guess we should change the install.py to use these > values instead of building its own little world of > pathnames. > > What do you think ? This makes a lot of sense. Anything that can prevent the ugly patches for multilib. Misa