From seefeld@sympatico.ca Thu Feb 1 11:43:16 2001 From: seefeld@sympatico.ca (Stefan Seefeld) Date: Thu Feb 1 11:43:16 2001 Subject: [Distutils] how to fine tune parameters for extension building Message-ID: <3A799F93.B73250D4@sympatico.ca> Hi there, I'm working on the 'synopsis' project, a source code inspection tool mainly developed in python, but with a number of extensions in C++. (http://Synopsis.sourceforge.net) I'm considering trying out the distutils, but I'm stumbling about a couple of (minor) issues. I don't seem to be able to specify compiler names and options, only the options as provided by the Extension dictionary itself. As I would replace all the Makefiles, I'd of course need the same degree of flexibility in choosing the tools I need, for example calling lex/bison, or whatever. May be distutils is currently not up to the task to replace make, in which case I would just stay with the current build system I have. Any help is highly appreciated ! Stefan From akuchlin@mems-exchange.org Thu Feb 1 12:01:52 2001 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Thu Feb 1 12:01:52 2001 Subject: [Distutils] Proposal: passing options to SWIG In-Reply-To: ; from seward@mza.com on Tue, Jan 30, 2001 at 09:25:55AM -0700 References: Message-ID: <20010201095554.A27394@newcnri.cnri.reston.va.us> On Tue, Jan 30, 2001 at 09:25:55AM -0700, Tony Seward wrote: >I would like to propose adding a command line option to build_ext that would >pass options to SWIG. The syntax would be something like >--swig-opts='-shadow'. I have some code working and I would like to get This rings a faint bell for me; is there already a patch on SourceForge to add similar functionality? --amk From mal@lemburg.com Thu Feb 1 17:03:43 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu Feb 1 17:03:43 2001 Subject: [Distutils] how to fine tune parameters for extension building References: <3A799F93.B73250D4@sympatico.ca> Message-ID: <3A79A69B.D1FC1C79@lemburg.com> Stefan Seefeld wrote: > > Hi there, > > I'm working on the 'synopsis' project, a source code inspection tool > mainly developed in python, but with a number of extensions in C++. > (http://Synopsis.sourceforge.net) > > I'm considering trying out the distutils, but I'm stumbling about > a couple of (minor) issues. I don't seem to be able to specify compiler > names and options, only the options as provided by the Extension dictionary > itself. As I would replace all the Makefiles, I'd of course need the > same degree of flexibility in choosing the tools I need, for example > calling lex/bison, or whatever. > > May be distutils is currently not up to the task to replace make, > in which case I would just stay with the current build system I have. > > Any help is highly appreciated ! You could write new commands and hook them into the build chain in order to run other currently unsupported tools like flex/yacc. Check out the autoconf example in the mxDateTime setup.py file provided in distutils. This shows how to add new commands and how to link them into the processing chain. Alternatively, take a look at this package ... http://www.lemburg.com/python/mx-Extensions-BASE-1.0.0.zip ... which does some of these things. -- Marc-Andre Lemburg ______________________________________________________________________ Company: http://www.egenix.com/ Consulting: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From seefeld@sympatico.ca Fri Feb 2 02:35:10 2001 From: seefeld@sympatico.ca (Stefan Seefeld) Date: Fri Feb 2 02:35:10 2001 Subject: [Distutils] how to fine tune parameters for extension building References: <3A799F93.B73250D4@sympatico.ca> <3A79A69B.D1FC1C79@lemburg.com> Message-ID: <3A7A4C49.1869C1C8@sympatico.ca> > Check out the autoconf example in the mxDateTime setup.py file > provided in distutils. This shows how to add new commands and > how to link them into the processing chain. ok, thanks. Trying this, however, I get errors from distutils.command.config, the _preprocess method is called with the wrong number of arguments. Regards, Stefan From robin@jessikat.fsnet.co.uk Fri Feb 2 05:09:01 2001 From: robin@jessikat.fsnet.co.uk (Robin Becker) Date: Fri Feb 2 05:09:01 2001 Subject: [Distutils] libraries options Message-ID: It seems that the libraries options are different to the Extension ones. is it intentional that macros <> define_macros and that there is no equivalent to undef_macros? -- Robin Becker From thomas.heller@ion-tof.com Fri Feb 2 08:32:01 2001 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Fri Feb 2 08:32:01 2001 Subject: [Distutils] libraries options References: Message-ID: <023201c08d1c$7092bf40$e000a8c0@thomasnotebook> Robin Becker writes > It seems that the libraries options are different to the Extension ones. > is it intentional that macros <> define_macros and that there is no > equivalent to undef_macros? I don't think so. To me it seemed strange that I have to use an instance of Extension to specify 'ext_modules', but have to supply dictionaries to 'libraries'. ext_modules and libraries probably should be unified. Thomas From HoumanG@ActiveState.com Fri Feb 2 13:13:04 2001 From: HoumanG@ActiveState.com (Houman G) Date: Fri Feb 2 13:13:04 2001 Subject: [Distutils] Removing Packages Message-ID: <000501c08d44$2643e450$ba03a8c0@activestate.ca> As far as I know Distutils does not have an 'uninstall command' (unless you are using the Windows installer for installing packages), nor there is standard way of removing packages. I was wondering if someone has come up with a mechanism for removing packages, or if there is a Distutils command for doing that. -- Houman Ghaemi houmang@ActiveState.com From thomas.heller@ion-tof.com Fri Feb 2 13:22:01 2001 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Fri Feb 2 13:22:01 2001 Subject: [Distutils] Removing Packages References: <000501c08d44$2643e450$ba03a8c0@activestate.ca> Message-ID: <04c201c08d44$f1f03e00$e000a8c0@thomasnotebook> > As far as I know Distutils does not have an 'uninstall command' (unless you > are using the Windows installer for installing packages), nor there is > standard way of removing packages. I was wondering if someone has come up > with a mechanism for removing packages, or if there is a Distutils command > for doing that. No, there is currently no support for that. Even bdist_wininst (if you mean that by Windows installer), currently is not able to do it, although this is on my to-do list. Thomas Heller From mal@lemburg.com Fri Feb 2 14:48:01 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri Feb 2 14:48:01 2001 Subject: [Distutils] Removing Packages References: <000501c08d44$2643e450$ba03a8c0@activestate.ca> Message-ID: <3A7B0E81.CCBABC3B@lemburg.com> Houman G wrote: > > As far as I know Distutils does not have an 'uninstall command' (unless you > are using the Windows installer for installing packages), nor there is > standard way of removing packages. I was wondering if someone has come up > with a mechanism for removing packages, or if there is a Distutils command > for doing that. The Windows doesn't do uninstall either... at least not yet. As for the rest of distutils, I think it would be a good idea to have distutils store information about which packages and files were installed in a database much like RPM does. To simplify things we could start with a pickled dictionary in a standard place, e.g. as file distutils.db in the standard installation location. I'd suggest to store objects in this dictionary to be able to easily upgrade the format. It should be indexed by package name for simplicity and since this will be the most used query form. What other information would we need apart from a list of files which were installed by the package and maybe its version number ? -- Marc-Andre Lemburg ______________________________________________________________________ Company: http://www.egenix.com/ Consulting: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From thomas.heller@ion-tof.com Fri Feb 2 15:15:01 2001 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Fri Feb 2 15:15:01 2001 Subject: [Distutils] Removing Packages References: <000501c08d44$2643e450$ba03a8c0@activestate.ca> <3A7B0E81.CCBABC3B@lemburg.com> Message-ID: <054b01c08d54$b0157760$e000a8c0@thomasnotebook> ----- Original Message ----- From: "M.-A. Lemburg" To: Cc: "'distutils-sig'" Sent: Friday, February 02, 2001 8:46 PM Subject: Re: [Distutils] Removing Packages > Houman G wrote: > > > > As far as I know Distutils does not have an 'uninstall command' (unless you > > are using the Windows installer for installing packages), nor there is > > standard way of removing packages. I was wondering if someone has come up > > with a mechanism for removing packages, or if there is a Distutils command > > for doing that. > > The Windows doesn't do uninstall either... at least not yet. > > As for the rest of distutils, I think it would be a good idea to > have distutils store information about which packages and files > were installed in a database much like RPM does. To simplify things > we could start with a pickled dictionary in a standard place, e.g. > as file distutils.db in the standard installation location. > > I'd suggest to store objects in this dictionary to be able to > easily upgrade the format. It should be indexed by package name > for simplicity and since this will be the most used query form. Michael Muller submitted a 'Package Meta Information Patch' back in december 1998. There was a lively discussion about this patch starting in jan 1999, also the data-format has been discussed. Executable python code was suggested, also ConfigParser format. Perhaps we should read this again, I cannot remember the conclusion. Thomas From uche.ogbuji@fourthought.com Sat Feb 3 17:53:01 2001 From: uche.ogbuji@fourthought.com (Uche Ogbuji) Date: Sat Feb 3 17:53:01 2001 Subject: [Distutils] Nasty typo in Python 2.1a2 distutils (with fix) Message-ID: <3A7C8BC0.78291120@fourthought.com> Sourceforge seems to be having trouble with bug submissions right now, si I'm just posting here. I'm not a subscriber, so cc me on any replies. There is a typo/bug in the Python 2.1a2 alpha version of python2.1/distutils/command/install_data.py that breaks the install_data command. See https://sourceforge.net/patch/?func=detailpatch&patch_id=103587&group_id=5470 For the simple patch. -- Uche Ogbuji Principal Consultant uche.ogbuji@fourthought.com +1 303 583 9900 x 101 Fourthought, Inc. http://Fourthought.com 4735 East Walnut St, Ste. C, Boulder, CO 80301-2537, USA Software-engineering, knowledge-management, XML, CORBA, Linux, Python From robin@jessikat.fsnet.co.uk Mon Feb 5 07:46:24 2001 From: robin@jessikat.fsnet.co.uk (Robin Becker) Date: Mon Feb 5 07:46:24 2001 Subject: [Distutils] libraries options In-Reply-To: <023201c08d1c$7092bf40$e000a8c0@thomasnotebook> References: <023201c08d1c$7092bf40$e000a8c0@thomasnotebook> Message-ID: In article <023201c08d1c$7092bf40$e000a8c0@thomasnotebook>, Thomas Heller writes >Robin Becker writes >> It seems that the libraries options are different to the Extension ones. >> is it intentional that macros <> define_macros and that there is no >> equivalent to undef_macros? >I don't think so. >To me it seemed strange that I have to use an instance of Extension >to specify 'ext_modules', but have to supply dictionaries to 'libraries'. > >ext_modules and libraries probably should be unified. > >Thomas > OK I have been using static libs with distutils for a couple of days and have noticed the following. 1) Under win32 at least the .lib files are placed by default in temp.win32-2.0\ imho they should be placed alongside other object modules. 2) The lib objects aren't placed in a release/debug subdirectory. 3) If a dynamic lib of the same name is present in one of the directories mentioned in a libpath option that lib gets used first (wrongly). -- Robin Becker From phrxy@csv.warwick.ac.uk Mon Feb 5 10:59:25 2001 From: phrxy@csv.warwick.ac.uk (John J. Lee) Date: Mon Feb 5 10:59:25 2001 Subject: [Distutils] libraries options In-Reply-To: Message-ID: On Mon, 5 Feb 2001, Robin Becker wrote: > In article <023201c08d1c$7092bf40$e000a8c0@thomasnotebook>, Thomas > Heller writes [...] > OK I have been using static libs with distutils for a couple of days and > have noticed the following. > > 1) Under win32 at least the .lib files are placed by default in > temp.win32-2.0\ imho they should be placed alongside other object > modules. That is mentioned in the source as being deliberate. May or may not be the 'right' place though, of course. John From robin@jessikat.fsnet.co.uk Mon Feb 5 11:59:27 2001 From: robin@jessikat.fsnet.co.uk (Robin Becker) Date: Mon Feb 5 11:59:27 2001 Subject: [Distutils] libraries options In-Reply-To: References: Message-ID: <8J1+FJAdutf6Ew+N@jessikat.fsnet.co.uk> In article , John J. Lee writes >On Mon, 5 Feb 2001, Robin Becker wrote: > >> In article <023201c08d1c$7092bf40$e000a8c0@thomasnotebook>, Thomas >> Heller writes >[...] >> OK I have been using static libs with distutils for a couple of days and >> have noticed the following. >> >> 1) Under win32 at least the .lib files are placed by default in >> temp.win32-2.0\ imho they should be placed alongside other object >> modules. > >That is mentioned in the source as being deliberate. May or may not be >the 'right' place though, of course. > > >John > > the problem is that I might have different things in a 'DEBUG' library. Having only one set of .objs and one .lib (or presumably .a) might muck up the nascent Release/Debug build ideas. -- Robin Becker From mal@lemburg.com Mon Feb 5 13:57:22 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Mon Feb 5 13:57:22 2001 Subject: [Distutils] libraries options References: <023201c08d1c$7092bf40$e000a8c0@thomasnotebook> Message-ID: <3A7EF75F.BE1B5E56@lemburg.com> Robin Becker wrote: > > In article <023201c08d1c$7092bf40$e000a8c0@thomasnotebook>, Thomas > Heller writes > >Robin Becker writes > >> It seems that the libraries options are different to the Extension ones. > >> is it intentional that macros <> define_macros and that there is no > >> equivalent to undef_macros? > >I don't think so. > >To me it seemed strange that I have to use an instance of Extension > >to specify 'ext_modules', but have to supply dictionaries to 'libraries'. > > > >ext_modules and libraries probably should be unified. > > > >Thomas > > > OK I have been using static libs with distutils for a couple of days and > have noticed the following. > > 1) Under win32 at least the .lib files are placed by default in > temp.win32-2.0\ imho they should be placed alongside other object > modules. > > 2) The lib objects aren't placed in a release/debug subdirectory. > > 3) If a dynamic lib of the same name is present in one of the > directories mentioned in a libpath option that lib gets used first > (wrongly). Hmm, the source says: # The MSVC linker generates .lib and .exp files, which cannot be # suppressed by any linker switches. The .lib files may even be # needed! Make sure they are generated in the temporary build # directory. Since they have different names for debug and release # builds, they can go into the same directory. Don't the debug builds have the "_d" suffix ? This should avoid any problems with mixing the two. About 3): if your import lib uses the same name as a common DLL then your lib should probably use a different name, since this would cause problems sooner or later anyway. Unless, of course, I've overlooked something here -- I'm not much of a Windows guy ;) -- Marc-Andre Lemburg ______________________________________________________________________ Company: http://www.egenix.com/ Consulting: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From robin@jessikat.fsnet.co.uk Tue Feb 6 04:59:32 2001 From: robin@jessikat.fsnet.co.uk (Robin Becker) Date: Tue Feb 6 04:59:32 2001 Subject: [Distutils] libraries options In-Reply-To: <3A7EF75F.BE1B5E56@lemburg.com> References: <023201c08d1c$7092bf40$e000a8c0@thomasnotebook> <3A7EF75F.BE1B5E56@lemburg.com> Message-ID: In article <3A7EF75F.BE1B5E56@lemburg.com>, M.-A. Lemburg writes >Robin Becker wrote: ..... >> OK I have been using static libs with distutils for a couple of days and >> have noticed the following. >> >> 1) Under win32 at least the .lib files are placed by default in >> temp.win32-2.0\ imho they should be placed alongside other object >> modules. >> >> 2) The lib objects aren't placed in a release/debug subdirectory. >> >> 3) If a dynamic lib of the same name is present in one of the >> directories mentioned in a libpath option that lib gets used first >> (wrongly). > >Hmm, the source says: > > # The MSVC linker generates .lib and .exp files, which cannot be > # suppressed by any linker switches. The .lib files may even be > # needed! Make sure they are generated in the temporary build > # directory. Since they have different names for debug and release > # builds, they can go into the same directory. > and for _ext builds they do indeed make these in the appropriate place eg building bingo.pyd build/temp.win32-2.0/Release contains bingo.obj, bingo.exp and bingo.lib and the .pyd ends up in build/lib.win32-2.0 however building static dingo.lib for use by bingo dingo.obj ends up in build/temp.win32-2.0/dingo and dingo.lib ends up in build/temp.win32-2.0/ Certainly if I build two .pyds each using different subroutines sub.c they're going to clash in Release. >Don't the debug builds have the "_d" suffix ? This should avoid >any problems with mixing the two. that may be true, but since we have only one .obj directory don't the objects clash? > >About 3): if your import lib uses the same name as a common >DLL then your lib should probably use a different name, since >this would cause problems sooner or later anyway. > Currently the linker is presented with the /libpath option before the objects. seems that M$'s linker is pretty straight forward if I give link xxx/release/aing.obj xxx/bing.lib /libpath:yyy .... then a bing.lib from yyy isn't consulted. This isn't safe of course, but it's clear that static libs which distutils builds should be presented along with objs before any lib paths and we don't need to use a /libpath to get to xxx it should be explicit. >Unless, of course, I've overlooked something here -- I'm not >much of a Windows guy ;) > -- Robin Becker From mal@lemburg.com Tue Feb 6 07:02:33 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Tue Feb 6 07:02:33 2001 Subject: [Distutils] libraries options References: <023201c08d1c$7092bf40$e000a8c0@thomasnotebook> <3A7EF75F.BE1B5E56@lemburg.com> Message-ID: <3A7FE794.C7EC9AD4@lemburg.com> Robin Becker wrote: > > In article <3A7EF75F.BE1B5E56@lemburg.com>, M.-A. Lemburg > writes > >Robin Becker wrote: > ..... > >> OK I have been using static libs with distutils for a couple of days and > >> have noticed the following. > >> > >> 1) Under win32 at least the .lib files are placed by default in > >> temp.win32-2.0\ imho they should be placed alongside other object > >> modules. > >> > >> 2) The lib objects aren't placed in a release/debug subdirectory. > >> > >> 3) If a dynamic lib of the same name is present in one of the > >> directories mentioned in a libpath option that lib gets used first > >> (wrongly). > > > >Hmm, the source says: > > > > # The MSVC linker generates .lib and .exp files, which cannot be > > # suppressed by any linker switches. The .lib files may even be > > # needed! Make sure they are generated in the temporary build > > # directory. Since they have different names for debug and release > > # builds, they can go into the same directory. > > > and for _ext builds they do indeed make these in the appropriate place > > eg building bingo.pyd > > build/temp.win32-2.0/Release > > contains bingo.obj, bingo.exp and bingo.lib and the .pyd ends up in > > build/lib.win32-2.0 > > however building static dingo.lib for use by bingo > > dingo.obj ends up in > > build/temp.win32-2.0/dingo and dingo.lib ends up in > build/temp.win32-2.0/ > > Certainly if I build two .pyds each using different subroutines sub.c > they're going to clash in Release. > > >Don't the debug builds have the "_d" suffix ? This should avoid > >any problems with mixing the two. > > that may be true, but since we have only one .obj directory don't the > objects clash? I guess so. Would it be hard to move all objects and libs into the Release/Debug dirs, rather than keeping them in the top-level build directory ? I am not sure about the consequences of making this change, but since we're in control here (distutils is building the command line, so we can fiddle with it in any way we want to), we can easily leverage any problems this might cause by adding some explicit paths. Could you submit a patch for this ? > > > >About 3): if your import lib uses the same name as a common > >DLL then your lib should probably use a different name, since > >this would cause problems sooner or later anyway. > > > > Currently the linker is presented with the /libpath option before the > objects. seems that M$'s linker is pretty straight forward if I give > > link xxx/release/aing.obj xxx/bing.lib /libpath:yyy .... > > then a bing.lib from yyy isn't consulted. This isn't safe of course, but > it's clear that static libs which distutils builds should be presented > along with objs before any lib paths and we don't need to use a /libpath > to get to xxx it should be explicit. > > >Unless, of course, I've overlooked something here -- I'm not > >much of a Windows guy ;) > > > > -- > Robin Becker > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG@python.org > http://mail.python.org/mailman/listinfo/distutils-sig -- Marc-Andre Lemburg ______________________________________________________________________ Company: http://www.egenix.com/ Consulting: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From robin@jessikat.fsnet.co.uk Tue Feb 6 07:37:33 2001 From: robin@jessikat.fsnet.co.uk (Robin Becker) Date: Tue Feb 6 07:37:33 2001 Subject: [Distutils] libraries options In-Reply-To: <3A7FE794.C7EC9AD4@lemburg.com> References: <023201c08d1c$7092bf40$e000a8c0@thomasnotebook> <3A7EF75F.BE1B5E56@lemburg.com> <3A7FE794.C7EC9AD4@lemburg.com> Message-ID: In article <3A7FE794.C7EC9AD4@lemburg.com>, M.-A. Lemburg writes >Robin Becker wrote: >> ..... .. .. > >> that may be true, but since we have only one .obj directory don't the >> objects clash? > >I guess so. > >Would it be hard to move all objects and libs into >the Release/Debug dirs, rather than keeping them in the top-level >build directory ? > >I am not sure about the consequences of making this change, >but since we're in control here (distutils is building the >command line, so we can fiddle with it in any way we want to), >we can easily leverage any problems this might cause by adding >some explicit paths. > .... The options stuff is really messy and hard; I'll have to get myself psyched up and do some debugging. -- Robin Becker From R.Liebscher@gmx.de Tue Feb 6 07:46:40 2001 From: R.Liebscher@gmx.de (Rene Liebscher) Date: Tue Feb 6 07:46:40 2001 Subject: [Distutils] libraries options References: <023201c08d1c$7092bf40$e000a8c0@thomasnotebook> <3A7EF75F.BE1B5E56@lemburg.com> <3A7FE794.C7EC9AD4@lemburg.com> Message-ID: <3A7FF1AC.F574442A@gmx.de> "M.-A. Lemburg" wrote: > > Robin Becker wrote: > > > > In article <3A7EF75F.BE1B5E56@lemburg.com>, M.-A. Lemburg > > writes > > >Robin Becker wrote: > > ..... > > >> OK I have been using static libs with distutils for a couple of days and > > >> have noticed the following. > > >> > > >> 1) Under win32 at least the .lib files are placed by default in > > >> temp.win32-2.0\ imho they should be placed alongside other object > > >> modules. > > >> > > >> 2) The lib objects aren't placed in a release/debug subdirectory. > > >> > > >> 3) If a dynamic lib of the same name is present in one of the > > >> directories mentioned in a libpath option that lib gets used first > > >> (wrongly). > > > > > >Hmm, the source says: > > > > > > # The MSVC linker generates .lib and .exp files, which cannot be > > > # suppressed by any linker switches. The .lib files may even be > > > # needed! Make sure they are generated in the temporary build > > > # directory. Since they have different names for debug and release > > > # builds, they can go into the same directory. > > > > > and for _ext builds they do indeed make these in the appropriate place > > > > eg building bingo.pyd > > > > build/temp.win32-2.0/Release > > > > contains bingo.obj, bingo.exp and bingo.lib and the .pyd ends up in > > > > build/lib.win32-2.0 > > > > however building static dingo.lib for use by bingo > > > > dingo.obj ends up in > > > > build/temp.win32-2.0/dingo and dingo.lib ends up in > > build/temp.win32-2.0/ > > > > Certainly if I build two .pyds each using different subroutines sub.c > > they're going to clash in Release. > > > > >Don't the debug builds have the "_d" suffix ? This should avoid > > >any problems with mixing the two. > > > > that may be true, but since we have only one .obj directory don't the > > objects clash? > > I guess so. > > Would it be hard to move all objects and libs into > the Release/Debug dirs, rather than keeping them in the top-level > build directory ? The difference between dynamic and static libraries is, that in build_ext the build_temp variable gets "Release" or "Debug" appended where build_clib doesn't change build_temp. If you would copy these (about 5) lines (from build_ext finalize_options) into build_clib.py your problems should disappear. (Maybe you have also to change the build_clib variable.) > I am not sure about the consequences of making this change, > but since we're in control here (distutils is building the > command line, so we can fiddle with it in any way we want to), > we can easily leverage any problems this might cause by adding > some explicit paths. > > Could you submit a patch for this ? > >... Kind regards Rene Liebscher From enrico.sirola@riskmap.net Wed Feb 7 13:21:52 2001 From: enrico.sirola@riskmap.net (Enrico Sirola) Date: Wed Feb 7 13:21:52 2001 Subject: [Distutils] changing default compiler Message-ID: <87itml69bh.fsf@sirola01.risk> Hello, i wrote a C++ extention module, but distutils keep trying to compile it as a plain C interface one. Is t possible it's confused by my strange platform name? from sys import platform print platform gives out linux2 This is prolly non standard, but maybe it's becouse i'm using a beta distro (progeny debian)... Anyway everithing works if i run compiler commands by hand (changing gcc to g++) and finally type "python setup.py install". So, how can i force distutils to use g++ instead of gcc? Thanks a lot in advance, Enrico From mal@lemburg.com Wed Feb 7 13:59:42 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Wed Feb 7 13:59:42 2001 Subject: [Distutils] changing default compiler References: <87itml69bh.fsf@sirola01.risk> Message-ID: <3A819ABC.5A4901E2@lemburg.com> Enrico Sirola wrote: > > Hello, > i wrote a C++ extention module, but distutils keep trying to compile > it as a plain C interface one. Is t possible it's confused by my > strange platform name? > > from sys import platform > print platform > > gives out > > linux2 > > This is prolly non standard, but maybe it's becouse i'm using a beta > distro (progeny debian)... > Anyway everithing works if i run compiler commands by hand (changing > gcc to g++) and finally type "python setup.py install". > So, how can i force distutils to use g++ instead of gcc? Try the --compiler option to the build_ext command. Note that build_ext is not yet up to compiling C++ code. For gcc a simple renaming of the source file to .cc or .cxx will let gcc compile the file as C++ without any additional options. -- Marc-Andre Lemburg ______________________________________________________________________ Company: http://www.egenix.com/ Consulting: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From enrico.sirola@riskmap.net Thu Feb 8 04:27:55 2001 From: enrico.sirola@riskmap.net (Enrico Sirola) Date: Thu Feb 8 04:27:55 2001 Subject: [Distutils] changing default compiler In-Reply-To: "M.-A. Lemburg"'s message of "Wed, 07 Feb 2001 19:58:04 +0100" References: <87itml69bh.fsf@sirola01.risk> <3A819ABC.5A4901E2@lemburg.com> Message-ID: <87snlo6g9f.fsf@sirola01.risk> >>>>> "MA" == M -A Lemburg writes: MA> Enrico Sirola wrote: >> Hello, i wrote a C++ extention module, but distutils keep >> trying to compile it as a plain C interface one. Is t possible >> it's confused by my strange platform name? >> >> from sys import platform print platform >> >> gives out >> >> linux2 >> >> This is prolly non standard, but maybe it's becouse i'm using a >> beta distro (progeny debian)... Anyway everithing works if i >> run compiler commands by hand (changing gcc to g++) and finally >> type "python setup.py install". So, how can i force distutils >> to use g++ instead of gcc? <*slap* self> I was wrong... on standard debian platforms python has the same behavior and everything works fine in distutils. MA> Try the --compiler option to the build_ext command. Note that MA> build_ext is not yet up to compiling C++ code. For gcc a MA> simple renaming of the source file to .cc or .cxx will let gcc MA> compile the file as C++ without any additional options. i just tried, but --compiler let me choose 'unix' only (and then it chooses gcc instead of g++). there's another strange behavior: the directories under build/ have the following names: lib.linux2/ temp.linux2/ and prolly should be lib.linux-i686-1.5/ temp.linux-i686-1.5/ How do distutils select buid-directory names? This could be a good starting point.. Thanks for the help Enrico -- From mal@lemburg.com Thu Feb 8 04:51:54 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu Feb 8 04:51:54 2001 Subject: [Distutils] changing default compiler References: <87itml69bh.fsf@sirola01.risk> <3A819ABC.5A4901E2@lemburg.com> <87snlo6g9f.fsf@sirola01.risk> Message-ID: <3A826B5F.31A7A232@lemburg.com> Enrico Sirola wrote: > > >>>>> "MA" == M -A Lemburg writes: > > MA> Enrico Sirola wrote: > >> Hello, i wrote a C++ extention module, but distutils keep > >> trying to compile it as a plain C interface one. Is t possible > >> it's confused by my strange platform name? > >> > >> from sys import platform print platform > >> > >> gives out > >> > >> linux2 > >> > >> This is prolly non standard, but maybe it's becouse i'm using a > >> beta distro (progeny debian)... Anyway everithing works if i > >> run compiler commands by hand (changing gcc to g++) and finally > >> type "python setup.py install". So, how can i force distutils > >> to use g++ instead of gcc? > > <*slap* self> > I was wrong... on standard debian platforms python has the same > behavior and everything works fine in distutils. > > MA> Try the --compiler option to the build_ext command. Note that > MA> build_ext is not yet up to compiling C++ code. For gcc a > MA> simple renaming of the source file to .cc or .cxx will let gcc > MA> compile the file as C++ without any additional options. > > i just tried, but --compiler let me choose 'unix' only (and then it > chooses gcc instead of g++). Dang, you're right. C++ support should go on the TODO list then... it should either auto-detect C vs. C++ depending on the filename or provide an option --c++ for explicitely using a C++ compiler. > there's another strange behavior: > the directories under build/ have the following names: > lib.linux2/ > temp.linux2/ > and prolly should be > lib.linux-i686-1.5/ > temp.linux-i686-1.5/ > How do distutils select buid-directory names? This could be a good > starting point.. Well, it uses distutils.util.get_platform() which uses sys.platform as fall-back on Unix if the os module doesn't define uname(). On SuSE I get correct names (linux-i586). Could you verify this ? -- Marc-Andre Lemburg ______________________________________________________________________ Company: http://www.egenix.com/ Consulting: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From thomas.heller@ion-tof.com Fri Feb 9 14:34:02 2001 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Fri Feb 9 14:34:02 2001 Subject: [Distutils] Do we need an install_clib command? Message-ID: <03ab01c092cf$35571940$e000a8c0@thomasnotebook> I recently came to think about an install_clib command. The current build_clib builds static libraries, but they can only be used to link them into extensions. Sometimes it may be usefull to have static libraries around. Thomas From mal@lemburg.com Fri Feb 9 14:57:02 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri Feb 9 14:57:02 2001 Subject: [Distutils] Do we need an install_clib command? References: <03ab01c092cf$35571940$e000a8c0@thomasnotebook> Message-ID: <3A844B4F.663DB76@lemburg.com> Thomas Heller wrote: > > I recently came to think about an install_clib command. > > The current build_clib builds static libraries, > but they can only be used to link them into extensions. > > Sometimes it may be usefull to have static libraries > around. True, but then we'd also have to support dynamic libs for completeness (even though I believe that tools like RPM do a better job at managing these). -- Marc-Andre Lemburg ______________________________________________________________________ Company: http://www.egenix.com/ Consulting: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From jeremy@alum.mit.edu Thu Feb 15 18:17:06 2001 From: jeremy@alum.mit.edu (Jeremy Hylton) Date: Thu Feb 15 18:17:06 2001 Subject: [Distutils] setup.py clean questions Message-ID: <14988.25377.589628.604170@w221.z064000254.bwi-md.dsl.cnc.net> I am still a relative novice with the distutils. I used the clean command today and was very confused by how it works. I don't know if my confusion is over the clean command or over the specific setup script. The setup.py was from Andrew's ZODB package. Let me know if my problem is specific to that package. This setup script is used for several packages, including C extensions and straight Python. When I run "python setup.py clean", I get a bunch of warnings, e.g. warning: clean: 'build/temp.linux-i686-2.1' does not exist - can't clean it I don't know what a build/temp.mumble is nor why I should care if it can't be cleaned. If it doesn't exist, it seems that I shouldn't have to be warned that it was cleaned up. Despite all these warnings, nothing has actually been cleaned up. The ExtensionClass build directory, which contains a bunch of .so files, still have all the .so files in it: ~/src/zodb> ls -R ExtensionClass/build/ ExtensionClass/build/: lib.linux-i686-2.1/ ExtensionClass/build/lib.linux-i686-2.1: Acquisition.so* MethodObject.so* MultiMapping.so* Sync.so* ExtensionClass.so* Missing.so* Record.so* ThreadLock.so* In yet another directory, I have some leftover build files from Python 2.0. There doesn't seem to be any way to remove these. A meta-question, of course, is what the current status of documentation is. I can't find any mention of the clean command in the documentation on python.org. Is anyone currently working on these? Late in the process, I decided to read the source and discovered a --all option to clean. That indeed got rid of most of the stuff I was interested in, but spewed even more warnings about directories that didn't exist. It did not, however, remove the 2.0 directories. Jeremy From jeremy.kloth@fourthought.com Thu Feb 15 18:48:59 2001 From: jeremy.kloth@fourthought.com (Jeremy Kloth) Date: Thu Feb 15 18:48:59 2001 Subject: [Distutils] setup.py clean questions References: <14988.25377.589628.604170@w221.z064000254.bwi-md.dsl.cnc.net> Message-ID: <3A8C6B5B.9D16370F@fourthought.com> Jeremy Hylton wrote: > Late in the process, I decided to read the source and discovered a > --all option to clean. That indeed got rid of most of the stuff I was > interested in, but spewed even more warnings about directories that > didn't exist. It did not, however, remove the 2.0 directories. Distutils only knows about the current version of Python that you run the setup.py script with. In the case of lib.linux-i686-2.1, it appears you are running Python 2.1, hence the -2.1 on the directory. So, if you want to clean 2.0 directories, run the setup.py script with Python 2.0. -- Jeremy Kloth Consultant jeremy.kloth@fourthought.com (303)583-9900 x 105 Fourthought, Inc. http://www.fourthought.com Software-engineering, knowledge-management, XML, CORBA, Linux, Python From jeremy@alum.mit.edu Thu Feb 15 19:00:05 2001 From: jeremy@alum.mit.edu (Jeremy Hylton) Date: Thu Feb 15 19:00:05 2001 Subject: [Distutils] setup.py clean questions In-Reply-To: <3A8C6B5B.9D16370F@fourthought.com> References: <14988.25377.589628.604170@w221.z064000254.bwi-md.dsl.cnc.net> <3A8C6B5B.9D16370F@fourthought.com> Message-ID: <14988.27981.464130.572451@w221.z064000254.bwi-md.dsl.cnc.net> >>>>> "JK" == Jeremy Kloth writes: JK> Jeremy Hylton wrote: >> Late in the process, I decided to read the source and discovered >> a --all option to clean. That indeed got rid of most of the >> stuff I was interested in, but spewed even more warnings about >> directories that didn't exist. It did not, however, remove the >> 2.0 directories. JK> Distutils only knows about the current version of Python that JK> you run the setup.py script with. In the case of JK> lib.linux-i686-2.1, it appears you are running Python 2.1, hence JK> the -2.1 on the directory. So, if you want to clean 2.0 JK> directories, run the setup.py script with Python 2.0. Perhaps there should be a realclean or distclean that actually removes all this stuff. I don't want to run setup.py twice with two different Python's just to clean up. Jeremy From robin@alldunn.com Thu Feb 15 20:14:01 2001 From: robin@alldunn.com (Robin Dunn) Date: Thu Feb 15 20:14:01 2001 Subject: [Distutils] setup.py clean questions References: <14988.25377.589628.604170@w221.z064000254.bwi-md.dsl.cnc.net><3A8C6B5B.9D16370F@fourthought.com> <14988.27981.464130.572451@w221.z064000254.bwi-md.dsl.cnc.net> Message-ID: <020001c097b5$7e572910$0100a8c0@Rogue> > > Perhaps there should be a realclean or distclean that actually removes > all this stuff. I don't want to run setup.py twice with two different > Python's just to clean up. > The command is already there, it's called "rm -rf build" -- Robin Dunn Software Craftsman robin@AllDunn.com Java give you jitters? http://wxPython.org Relax with wxPython! From mal@lemburg.com Fri Feb 16 04:27:00 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri Feb 16 04:27:00 2001 Subject: [Distutils] setup.py clean questions References: <14988.25377.589628.604170@w221.z064000254.bwi-md.dsl.cnc.net> <3A8C6B5B.9D16370F@fourthought.com> <14988.27981.464130.572451@w221.z064000254.bwi-md.dsl.cnc.net> Message-ID: <3A8CF22E.F73B2F18@lemburg.com> Jeremy Hylton wrote: > > >>>>> "JK" == Jeremy Kloth writes: > > JK> Jeremy Hylton wrote: > > >> Late in the process, I decided to read the source and discovered > >> a --all option to clean. That indeed got rid of most of the > >> stuff I was interested in, but spewed even more warnings about > >> directories that didn't exist. It did not, however, remove the > >> 2.0 directories. > > JK> Distutils only knows about the current version of Python that > JK> you run the setup.py script with. In the case of > JK> lib.linux-i686-2.1, it appears you are running Python 2.1, hence > JK> the -2.1 on the directory. So, if you want to clean 2.0 > JK> directories, run the setup.py script with Python 2.0. > > Perhaps there should be a realclean or distclean that actually removes > all this stuff. I don't want to run setup.py twice with two different > Python's just to clean up. Good point, but unfortunately difficult to implement since the clean command gets its parameter from the build command and you can easily override the names of the various build directories on the command line or via setup.cfg. OTOH, the clean could simply *try* to remove the directory trees using the default names and produce a warning in case this fails. We'd then need to expose the temp name generation logic in the build command, e.g. 'rm -rf temp.-* lib.-*' should do the trick. Alternatively, the clean command could simply remove the complete build_base tree... -- Marc-Andre Lemburg ______________________________________________________________________ Company: http://www.egenix.com/ Consulting: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From thomas.heller@ion-tof.com Fri Feb 16 04:34:59 2001 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Fri Feb 16 04:34:59 2001 Subject: [Distutils] setup.py clean questions References: <14988.25377.589628.604170@w221.z064000254.bwi-md.dsl.cnc.net> Message-ID: <026e01c097fb$940adfe0$e000a8c0@thomasnotebook> > A meta-question, of course, is what the current status of > documentation is. I can't find any mention of the clean command in > the documentation on python.org. Is anyone currently working on > these? > Not currently. The situation is desperate. We're all waiting for Greg. There are a lot of other commands not documented at all, only the online help 'setup.py install_data --help'. Even worse: Several options to the setup function are not mentioned anywhere. Although I'm not the right person to write docs (not beeing a native english speaker), and I don't have any qualities as author, I don't have any Latex experience, I'm in the process of convincing me to at least write _some_ docs for some of these. Thomas From jack@oratrix.nl Fri Feb 16 05:06:01 2001 From: jack@oratrix.nl (Jack Jansen) Date: Fri Feb 16 05:06:01 2001 Subject: [Distutils] setup.py clean questions In-Reply-To: Message by "M.-A. Lemburg" , Fri, 16 Feb 2001 10:26:06 +0100 , <3A8CF22E.F73B2F18@lemburg.com> Message-ID: <20010216100517.B4610371690@snelboot.oratrix.nl> > Good point, but unfortunately difficult to implement since the > clean command gets its parameter from the build command and > you can easily override the names of the various build directories > on the command line or via setup.cfg. > > OTOH, the clean could simply *try* to remove the directory > trees using the default names and produce a warning in case > this fails. We'd then need to expose the temp name generation logic > in the build command, e.g. 'rm -rf temp.-* lib.-*' > should do the trick. Alternatively, the clean command could simply > remove the complete build_base tree... Well, Jeremy was asking for a distclean-like facility, so I think that that is the solution. And, while we're at it, I wouldn't mind an uninstall either. -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm From mal@lemburg.com Fri Feb 16 05:12:59 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri Feb 16 05:12:59 2001 Subject: [Distutils] setup.py clean questions + uninstall References: <20010216100517.B4610371690@snelboot.oratrix.nl> Message-ID: <3A8CFCEC.1202CBDC@lemburg.com> Jack Jansen wrote: > > > Good point, but unfortunately difficult to implement since the > > clean command gets its parameter from the build command and > > you can easily override the names of the various build directories > > on the command line or via setup.cfg. > > > > OTOH, the clean could simply *try* to remove the directory > > trees using the default names and produce a warning in case > > this fails. We'd then need to expose the temp name generation logic > > in the build command, e.g. 'rm -rf temp.-* lib.-*' > > should do the trick. Alternatively, the clean command could simply > > remove the complete build_base tree... > > Well, Jeremy was asking for a distclean-like facility, so I think that that is > the solution. Ok, should we modify the clean command's --all option to do a rm -rf build then ? > And, while we're at it, I wouldn't mind an uninstall either. Patches appreciated ;-) Seriously, I think this needs a PEP since it targets a very special area: that of maintaining a Python (installation) registry. Jython already has something like this, so maybe we can learn from it for CPython... ideally, the registry should live in the config dir of the Python installation (could produce problems with file permissions though, e.g. in case someone wants to install a package in a private dir). -- Marc-Andre Lemburg ______________________________________________________________________ Company: http://www.egenix.com/ Consulting: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From thomas.heller@ion-tof.com Fri Feb 16 05:20:01 2001 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Fri Feb 16 05:20:01 2001 Subject: deinstall (Was: Re: [Distutils] setup.py clean questions ) References: <20010216100517.B4610371690@snelboot.oratrix.nl> Message-ID: <028901c09801$e305a0c0$e000a8c0@thomasnotebook> > And, while we're at it, I wouldn't mind an uninstall either. Here is a simple deinstall command, which you can use in your setup script. This can at least be a base for a discussion about a more complete one. It simply does the reverse of install. class deinstall(Command): description = "Remove all installed files" user_options = [] def initialize_options(self): pass def finalize_options(self): pass def run(self): self.run_command('build') build = self.get_finalized_command('build') install = self.get_finalized_command('install') self.announce("removing files") for n in 'platlib', 'purelib', 'headers', 'scripts', 'data': dstdir = getattr(install, 'install_' + n) try: srcdir = getattr(build, 'build_' + n) except AttributeError: pass else: self._removefiles(dstdir, srcdir) def _removefiles(self, dstdir, srcdir): if not os.path.isdir(srcdir): return for n in os.listdir(srcdir): name = os.path.join(dstdir, n) if os.path.isfile(name): self.announce("removing '%s'" % name) if not self.dry_run: os.remove(name) if os.path.splitext(name)[1] == '.py': try: os.remove(name + 'c') except: pass try: os.remove(name + 'o') except: pass elif os.path.isdir(name): self._removefiles(name, os.path.join(srcdir, n)) if not self.dry_run: os.rmdir(name) else: self.announce("skipping removal of '%s' (does not exist)" %\ name) ############################################################################ setup(..., cmdclass = {'deinstall': deinstall}, ...) Thomas From jack@oratrix.nl Fri Feb 16 05:36:01 2001 From: jack@oratrix.nl (Jack Jansen) Date: Fri Feb 16 05:36:01 2001 Subject: deinstall (Was: Re: [Distutils] setup.py clean questions ) In-Reply-To: Message by "Thomas Heller" , Fri, 16 Feb 2001 11:19:03 +0100 , <028901c09801$e305a0c0$e000a8c0@thomasnotebook> Message-ID: <20010216103506.05952371690@snelboot.oratrix.nl> > Here is a simple deinstall command, which you can > use in your setup script. This can at least be a base for > a discussion about a more complete one. Yes, this is pretty much what I had in mind. One point: > elif os.path.isdir(name): > self._removefiles(name, os.path.join(srcdir, n)) > if not self.dry_run: > os.rmdir(name) I think this needs a try/except around the rmdir(), with a message in the except that the rmdir failed, and possibly, if it failed because the directory wasn't empty, a note that there may may be user additions there. -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm From thomas.heller@ion-tof.com Fri Feb 16 05:47:02 2001 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Fri Feb 16 05:47:02 2001 Subject: deinstall (Was: Re: [Distutils] setup.py clean questions ) References: <20010216103506.05952371690@snelboot.oratrix.nl> Message-ID: <030b01c09805$af7cbd20$e000a8c0@thomasnotebook> > > > Here is a simple deinstall command, which you can > > use in your setup script. This can at least be a base for > > a discussion about a more complete one. > > Yes, this is pretty much what I had in mind. One point: > > > elif os.path.isdir(name): > > self._removefiles(name, os.path.join(srcdir, n)) > > if not self.dry_run: > > os.rmdir(name) > > I think this needs a try/except around the rmdir(), with a message in the > except that the rmdir failed, and possibly, if it failed because the directory > wasn't empty, a note that there may may be user additions there. OK. I currently only use this during development. Do you think we should try to establish this as a standard distutils command? Thomas From mal@lemburg.com Fri Feb 16 06:00:02 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri Feb 16 06:00:02 2001 Subject: deinstall (Was: Re: [Distutils] setup.py clean questions ) References: <20010216103506.05952371690@snelboot.oratrix.nl> <030b01c09805$af7cbd20$e000a8c0@thomasnotebook> Message-ID: <3A8D07FC.D1635F2A@lemburg.com> Thomas Heller wrote: > > > > > > Here is a simple deinstall command, which you can > > > use in your setup script. This can at least be a base for > > > a discussion about a more complete one. > > > > Yes, this is pretty much what I had in mind. One point: > > > > > elif os.path.isdir(name): > > > self._removefiles(name, os.path.join(srcdir, n)) > > > if not self.dry_run: > > > os.rmdir(name) > > > > I think this needs a try/except around the rmdir(), with a message in the > > except that the rmdir failed, and possibly, if it failed because the directory > > wasn't empty, a note that there may may be user additions there. > OK. > > I currently only use this during development. Do you think > we should try to establish this as a standard distutils command? It sure is a good start and far less contrversial than the RPM- style package repository which was discussed here a few times already. I would name the command 'uninstall' though, since this is the common term for these kinds of operations. One other area where try-except might be useful is around os.remove(): the user should be informed about the failure to complete the command in a different way than by a traceback... e.g. if the file is locked. If it cannot be found, I think the correct procedure should be to continue the command ignoring the error. -- Marc-Andre Lemburg ______________________________________________________________________ Company: http://www.egenix.com/ Consulting: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From thomas.heller@ion-tof.com Fri Feb 16 06:14:01 2001 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Fri Feb 16 06:14:01 2001 Subject: deinstall (Was: Re: [Distutils] setup.py clean questions ) References: <20010216103506.05952371690@snelboot.oratrix.nl> <030b01c09805$af7cbd20$e000a8c0@thomasnotebook> <3A8D07FC.D1635F2A@lemburg.com> Message-ID: <037401c09809$86d57e80$e000a8c0@thomasnotebook> > I would name the command 'uninstall' though, since this is the > common term for these kinds of operations. > > One other area where try-except might be useful is around > os.remove(): the user should be informed about the failure to > complete the command in a different way than by a traceback... > e.g. if the file is locked. If it cannot be found, I think the > correct procedure should be to continue the command ignoring > the error. Sure. New version: class uninstall(Command): description = "Remove all installed files" user_options = [] def initialize_options(self): pass def finalize_options(self): pass def run(self): self.run_command('build') build = self.get_finalized_command('build') install = self.get_finalized_command('install') self.announce("removing files") for n in 'platlib', 'purelib', 'headers', 'scripts', 'data': dstdir = getattr(install, 'install_' + n) try: srcdir = getattr(build, 'build_' + n) except AttributeError: pass else: self._removefiles(dstdir, srcdir) def _removefiles(self, dstdir, srcdir): if not os.path.isdir(srcdir): return for n in os.listdir(srcdir): name = os.path.join(dstdir, n) if os.path.isfile(name): self.announce("removing '%s'" % name) if not self.dry_run: try: os.remove(name) except OSError, details: self.warn("Could not remove file: %s" % details) if os.path.splitext(name)[1] == '.py': # Try to remove .pyc and -pyo files also try: os.remove(name + 'c') except OSError: pass try: os.remove(name + 'o') except OSError: pass elif os.path.isdir(name): self._removefiles(name, os.path.join(srcdir, n)) if not self.dry_run: try: os.rmdir(name) except OSError, details: self.warn("Are there additional user files?\n"\ " Could not remove directory: %s" % details) else: self.announce("skipping removal of '%s' (does not exist)" %\ name) From mal@lemburg.com Fri Feb 16 08:05:11 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri Feb 16 08:05:11 2001 Subject: deinstall (Was: Re: [Distutils] setup.py clean questions ) References: <20010216103506.05952371690@snelboot.oratrix.nl> <030b01c09805$af7cbd20$e000a8c0@thomasnotebook> <3A8D07FC.D1635F2A@lemburg.com> <037401c09809$86d57e80$e000a8c0@thomasnotebook> Message-ID: <3A8D251D.B60856A9@lemburg.com> Thomas Heller wrote: > > > I would name the command 'uninstall' though, since this is the > > common term for these kinds of operations. > > > > One other area where try-except might be useful is around > > os.remove(): the user should be informed about the failure to > > complete the command in a different way than by a traceback... > > e.g. if the file is locked. If it cannot be found, I think the > > correct procedure should be to continue the command ignoring > > the error. > Sure. > > New version: ... This works fine except that it does not remove data files which are installed in subdirs of the installation, e.g. documentation, READMEs and license information: data_files = [ 'mx/Doc/mxExtensions.html', 'mx/Doc/mxLicense.html', 'mx/DateTime/Doc/mxDateTime.html', 'mx/DateTime/Doc/mxDateTime-History.html', 'mx/DateTime/Doc/mxLicense.html', 'mx/DateTime/COPYRIGHT', 'mx/DateTime/LICENSE', ] Is there some easy way to get at those files from within the uninstall command ? -- Marc-Andre Lemburg ______________________________________________________________________ Company: http://www.egenix.com/ Consulting: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From R.Liebscher@gmx.de Fri Feb 16 08:52:02 2001 From: R.Liebscher@gmx.de (Rene Liebscher) Date: Fri Feb 16 08:52:02 2001 Subject: deinstall (Was: Re: [Distutils] setup.py clean questions ) References: <20010216103506.05952371690@snelboot.oratrix.nl> <030b01c09805$af7cbd20$e000a8c0@thomasnotebook> <3A8D07FC.D1635F2A@lemburg.com> <037401c09809$86d57e80$e000a8c0@thomasnotebook> <3A8D251D.B60856A9@lemburg.com> Message-ID: <3A8D2FFF.299A618A@gmx.de> "M.-A. Lemburg" wrote: > > Thomas Heller wrote: > > > > ... > > > > New version: ... > > This works fine except that it does not remove data files > which are installed in subdirs of the installation, e.g. > documentation, READMEs and license information: > > data_files = [ > > 'mx/Doc/mxExtensions.html', > 'mx/Doc/mxLicense.html', > > 'mx/DateTime/Doc/mxDateTime.html', > 'mx/DateTime/Doc/mxDateTime-History.html', > 'mx/DateTime/Doc/mxLicense.html', > 'mx/DateTime/COPYRIGHT', > 'mx/DateTime/LICENSE', > > ] > > Is there some easy way to get at those files from within the > uninstall command ? One could try to let the install(_data) command run in a quiet dry_run mode, afterwards you could get all file names by calling get_outputs() for the install command. (You have to run the command because at least for install_data the internal outputs variable is set by run(). ) Kind regards Rene Liebscher From thomas.heller@ion-tof.com Fri Feb 16 09:23:01 2001 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Fri Feb 16 09:23:01 2001 Subject: deinstall (Was: Re: [Distutils] setup.py clean questions ) References: <20010216103506.05952371690@snelboot.oratrix.nl> <030b01c09805$af7cbd20$e000a8c0@thomasnotebook> <3A8D07FC.D1635F2A@lemburg.com> <037401c09809$86d57e80$e000a8c0@thomasnotebook> <3A8D251D.B60856A9@lemburg.com> <3A8D2FFF.299A618A@gmx.de> Message-ID: <045b01c09823$ebae9b60$e000a8c0@thomasnotebook> > One could try to let the install(_data) command run in a quiet dry_run > mode, > afterwards you could get all file names by calling get_outputs() for > the install command. > (You have to run the command because at least for install_data the > internal outputs variable is set by run(). ) Better would be to replicate and invert the code from install_data.run(), because we would want to also remove directories created. Thomas From mal@lemburg.com Fri Feb 16 09:48:01 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri Feb 16 09:48:01 2001 Subject: deinstall (Was: Re: [Distutils] setup.py clean questions ) References: <20010216103506.05952371690@snelboot.oratrix.nl> <030b01c09805$af7cbd20$e000a8c0@thomasnotebook> <3A8D07FC.D1635F2A@lemburg.com> <037401c09809$86d57e80$e000a8c0@thomasnotebook> <3A8D251D.B60856A9@lemburg.com> <3A8D2FFF.299A618A@gmx.de> <045b01c09823$ebae9b60$e000a8c0@thomasnotebook> Message-ID: <3A8D3D4C.4A0070E4@lemburg.com> Thomas Heller wrote: > > > One could try to let the install(_data) command run in a quiet dry_run > > mode, > > afterwards you could get all file names by calling get_outputs() for > > the install command. > > (You have to run the command because at least for install_data the > > internal outputs variable is set by run(). ) > Better would be to replicate and invert the code from > install_data.run(), because we would want to also > remove directories created. Hmm, it would probably be better to add a new method to Commands in general which undoes the actions applied by .run(). In the above case, I'm using my own install_data Command class because I found the standard version unusable for my purposes (the standard command only installs data files in one directory, whereas I must install files in multiple directories). The uninstall command would then call the .uninstall() method of all install_* subcommands and have them do whatever is necessary to implement the uninstall machinery. -- Marc-Andre Lemburg ______________________________________________________________________ Company: http://www.egenix.com/ Consulting: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From thomas.heller@ion-tof.com Fri Feb 16 09:56:13 2001 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Fri Feb 16 09:56:13 2001 Subject: deinstall (Was: Re: [Distutils] setup.py clean questions ) References: <20010216103506.05952371690@snelboot.oratrix.nl> <030b01c09805$af7cbd20$e000a8c0@thomasnotebook> <3A8D07FC.D1635F2A@lemburg.com> <037401c09809$86d57e80$e000a8c0@thomasnotebook> <3A8D251D.B60856A9@lemburg.com> <3A8D2FFF.299A618A@gmx.de> <045b01c09823$ebae9b60$e000a8c0@thomasnotebook> <3A8D3D4C.4A0070E4@lemburg.com> Message-ID: <04d601c09828$8af35310$e000a8c0@thomasnotebook> Marc-Andre Lemburg: > > Hmm, it would probably be better to add a new method to Commands > in general which undoes the actions applied by .run(). > > In the above case, I'm using my own install_data Command class > because I found the standard version unusable for my purposes > (the standard command only installs data files in one directory, > whereas I must install files in multiple directories). > > The uninstall command would then call the .uninstall() method of > all install_* subcommands and have them do whatever is necessary > to implement the uninstall machinery. This would be one solution once the .uninstall() methods are all implemented. On the other hand, the install_data command is _completely_ undocumented (you have to read the source), and thus cannot really be improved. I'm quote sure there are missing some features... The 2.1 idle setup() script even uses it's own subclasses of build_py and install_lib only to copy some .txt and -gif files. Strange. Thomas From thomas.heller@ion-tof.com Fri Feb 16 10:03:08 2001 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Fri Feb 16 10:03:08 2001 Subject: deinstall (Was: Re: [Distutils] setup.py clean questions ) References: <20010216103506.05952371690@snelboot.oratrix.nl> <030b01c09805$af7cbd20$e000a8c0@thomasnotebook> <3A8D07FC.D1635F2A@lemburg.com> <037401c09809$86d57e80$e000a8c0@thomasnotebook> <3A8D251D.B60856A9@lemburg.com> <3A8D2FFF.299A618A@gmx.de> <045b01c09823$ebae9b60$e000a8c0@thomasnotebook> <3A8D3D4C.4A0070E4@lemburg.com> Message-ID: <04f501c09829$61d90eb0$e000a8c0@thomasnotebook> mal@lemburg.com: > data_files = [ > > 'mx/Doc/mxExtensions.html', > 'mx/Doc/mxLicense.html', > > 'mx/DateTime/Doc/mxDateTime.html', > 'mx/DateTime/Doc/mxDateTime-History.html', > 'mx/DateTime/Doc/mxLicense.html', > 'mx/DateTime/COPYRIGHT', > 'mx/DateTime/LICENSE', > > ] You can probably do something like this: data_files = [('mx/Doc', glob.glob('mx/Doc/mx*.html'), ('mx/DateTime/Doc', glob.glob('mx*.html')] to install into several subdirectories with the _standard_ install_data command. Thomas From R.Liebscher@gmx.de Fri Feb 16 10:18:01 2001 From: R.Liebscher@gmx.de (Rene Liebscher) Date: Fri Feb 16 10:18:01 2001 Subject: deinstall (Was: Re: [Distutils] setup.py clean questions ) References: <20010216103506.05952371690@snelboot.oratrix.nl> <030b01c09805$af7cbd20$e000a8c0@thomasnotebook> <3A8D07FC.D1635F2A@lemburg.com> <037401c09809$86d57e80$e000a8c0@thomasnotebook> <3A8D251D.B60856A9@lemburg.com> <3A8D2FFF.299A618A@gmx.de> <045b01c09823$ebae9b60$e000a8c0@thomasnotebook> <3A8D3D4C.4A0070E4@lemburg.com> <04d601c09828$8af35310$e000a8c0@thomasnotebook> Message-ID: <3A8D444F.5FF55F8A@gmx.de> Thomas Heller wrote: > > Marc-Andre Lemburg: > > > > Hmm, it would probably be better to add a new method to Commands > > in general which undoes the actions applied by .run(). > > > > In the above case, I'm using my own install_data Command class > > because I found the standard version unusable for my purposes > > (the standard command only installs data files in one directory, > > whereas I must install files in multiple directories). > > > > The uninstall command would then call the .uninstall() method of > > all install_* subcommands and have them do whatever is necessary > > to implement the uninstall machinery. > This would be one solution once the .uninstall() methods are all > implemented. > > On the other hand, the install_data command is _completely_ > undocumented (you have to read the source), and thus cannot really > be improved. I'm quote sure there are missing some features... > > The 2.1 idle setup() script even uses it's own subclasses > of build_py and install_lib only to copy some > .txt and -gif files. Strange. Own classes for install are not a problem, if they are build correct. For example bdist_rpm uses get_outputs() and if one has build its own subclasses so bdist_rpm still works, then you can also get the complete list of files from get_outputs(). For PyOpenGL I use a really complex install_data replacement and it would be easier to use it as it is than to extend it and later to explain the people which uses a later version of distutils (with install_data.uninstall()) why they see a traceback when they try to uninstall an older version of PyOpenGL. (BTW, not only PyOpenGL uses this install_data replacement, you can find it also in pyxml.) If you really want to extend install_* then you could use this at least as a kind of fallback if you try to uninstall an older package. Kind regards Rene Liebscher From thomas.heller@ion-tof.com Fri Feb 16 10:28:01 2001 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Fri Feb 16 10:28:01 2001 Subject: deinstall (Was: Re: [Distutils] setup.py clean questions ) References: <20010216103506.05952371690@snelboot.oratrix.nl> <030b01c09805$af7cbd20$e000a8c0@thomasnotebook> <3A8D07FC.D1635F2A@lemburg.com> <037401c09809$86d57e80$e000a8c0@thomasnotebook> <3A8D251D.B60856A9@lemburg.com> <3A8D2FFF.299A618A@gmx.de> <045b01c09823$ebae9b60$e000a8c0@thomasnotebook> <3A8D3D4C.4A0070E4@lemburg.com> <04d601c09828$8af35310$e000a8c0@thomasnotebook> <3A8D444F.5FF55F8A@gmx.de> Message-ID: <053f01c0982c$e323a180$e000a8c0@thomasnotebook> Rene Liebscher wrote: > Own classes for install are not a problem, if they are build correct. > For example bdist_rpm uses get_outputs() and if one has build its own > subclasses so bdist_rpm still works, then you can also get the complete > list of files from get_outputs(). > > For PyOpenGL I use a really complex install_data replacement and it > would be > easier to use it as it is than to extend it and later to explain the > people > which uses a later version of distutils (with install_data.uninstall()) > why > they see a traceback when they try to uninstall an older version of > PyOpenGL. > (BTW, not only PyOpenGL uses this install_data replacement, you can find > it > also in pyxml.) > > If you really want to extend install_* then you could use this at least > as > a kind of fallback if you try to uninstall an older package. > OK, sounds reasonable. So we would run install_data (or even install) in dry_run mode (or maybe even in not dry_run mode), and the use get_outputs() to find the files to delete. Why is install_data special: Because it does not copy the files into the build-directory first. Is this is design flaw? Thomas From jack@oratrix.nl Fri Feb 16 10:31:19 2001 From: jack@oratrix.nl (Jack Jansen) Date: Fri Feb 16 10:31:19 2001 Subject: deinstall (Was: Re: [Distutils] setup.py clean questions ) In-Reply-To: Message by "Thomas Heller" , Fri, 16 Feb 2001 16:01:46 +0100 , <04f501c09829$61d90eb0$e000a8c0@thomasnotebook> Message-ID: <20010216152958.5A218371690@snelboot.oratrix.nl> > You can probably do something like this: > > data_files = [('mx/Doc', glob.glob('mx/Doc/mx*.html'), > ('mx/DateTime/Doc', glob.glob('mx*.html')] > > to install into several subdirectories with the _standard_ > install_data command. NONONONONO, please! Not even more unixisms in distutils! The names you use in your setup script are URLs, not filenames. They look an awful lot like filenames on Unix, but they _are_ URLs. This is important on the Mac, where they're actually converted to the Mac filename convention. Using glob.glob on URLs will prove to be a dissapointing exercise there... Now if we could get some code in distutils that will make people's setup.py script remove itself if they put pathnames like "/usr/local/lib" in there... -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm From thomas.heller@ion-tof.com Fri Feb 16 10:42:00 2001 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Fri Feb 16 10:42:00 2001 Subject: deinstall (Was: Re: [Distutils] setup.py clean questions ) References: <20010216152958.5A218371690@snelboot.oratrix.nl> Message-ID: <057301c0982e$e8490130$e000a8c0@thomasnotebook> Jack Jansen: > NONONONONO, please! Not even more unixisms in distutils! The names you use in > your setup script are URLs, not filenames. They look an awful lot like > filenames on Unix, but they _are_ URLs. This is important on the Mac, where > they're actually converted to the Mac filename convention. Using glob.glob on > URLs will prove to be a dissapointing exercise there... > So how should they be written? glob.glob(os.path.join('mx', 'Doc', '*.html')) Thomas From jeremy@alum.mit.edu Fri Feb 16 10:45:05 2001 From: jeremy@alum.mit.edu (Jeremy Hylton) Date: Fri Feb 16 10:45:05 2001 Subject: [Distutils] setup.py clean questions In-Reply-To: <599461588@toto.iv> Message-ID: <14988.30230.467647.418579@w221.z064000254.bwi-md.dsl.cnc.net> >>>>> "TH" == Thomas Heller writes: >> A meta-question, of course, is what the current status of >> documentation is. I can't find any mention of the clean command >> in the documentation on python.org. Is anyone currently working >> on these? >> TH> Not currently. The situation is desperate. We're all waiting TH> for Greg. TH> There are a lot of other commands not documented at all, only TH> the online help 'setup.py install_data --help'. TH> Even worse: Several options to the setup function are not TH> mentioned anywhere. TH> Although I'm not the right person to write docs (not beeing a TH> native english speaker), and I don't have any qualities as TH> author, I don't have any Latex experience, I'm in the process of TH> convincing me to at least write _some_ docs for some of these. Is there anything I can do to help convince you? I do not follow this list closely and only understand how to do very simple things with distutils. I've been waiting for documentation to learn the rest. I'm sure there are lots of people who can help with markup and copy editing, but fewer who can describe what the code actually does. Jeremy From thomas.heller@ion-tof.com Fri Feb 16 11:12:00 2001 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Fri Feb 16 11:12:00 2001 Subject: [Distutils] setup.py clean questions References: <14988.30230.467647.418579@w221.z064000254.bwi-md.dsl.cnc.net> Message-ID: <05c201c09833$1653ab30$e000a8c0@thomasnotebook> Jeremy: > >>>>> "TH" == Thomas Heller writes: > > >> A meta-question, of course, is what the current status of > >> documentation is. I can't find any mention of the clean command > >> in the documentation on python.org. Is anyone currently working > >> on these? > >> > TH> Not currently. The situation is desperate. We're all waiting > TH> for Greg. > > TH> There are a lot of other commands not documented at all, only > TH> the online help 'setup.py install_data --help'. > > TH> Even worse: Several options to the setup function are not > TH> mentioned anywhere. > > TH> Although I'm not the right person to write docs (not beeing a > TH> native english speaker), and I don't have any qualities as > TH> author, I don't have any Latex experience, I'm in the process of > TH> convincing me to at least write _some_ docs for some of these. > > Is there anything I can do to help convince you? I do not follow this > list closely and only understand how to do very simple things with > distutils. I've been waiting for documentation to learn the rest. > > I'm sure there are lots of people who can help with markup and copy > editing, but fewer who can describe what the code actually does. > > Jeremy Maybe we can join forces here on distutils and post documentation, other distutil developers can comment and improve it? Thomas From mal@lemburg.com Fri Feb 16 11:21:21 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri Feb 16 11:21:21 2001 Subject: deinstall (Was: Re: [Distutils] setup.py clean questions ) References: <20010216152958.5A218371690@snelboot.oratrix.nl> Message-ID: <3A8D52E5.11E258CD@lemburg.com> Jack Jansen wrote: > > > You can probably do something like this: > > > > data_files = [('mx/Doc', glob.glob('mx/Doc/mx*.html'), > > ('mx/DateTime/Doc', glob.glob('mx*.html')] > > > > to install into several subdirectories with the _standard_ > > install_data command. > > NONONONONO, please! Not even more unixisms in distutils! The names you use in > your setup script are URLs, not filenames. They look an awful lot like > filenames on Unix, but they _are_ URLs. This is important on the Mac, where > they're actually converted to the Mac filename convention. Using glob.glob on > URLs will prove to be a dissapointing exercise there... > > Now if we could get some code in distutils that will make people's setup.py > script remove itself if they put pathnames like "/usr/local/lib" in there... But doesn't distutils internally use Unix file name conventions ? I don't think it is very useful to go about and try to write portable *configuration* files. After all, we have Python at hand, so let distutils do the job of converting the Unixisms into the target platform's conventions. -- Marc-Andre Lemburg ______________________________________________________________________ Company: http://www.egenix.com/ Consulting: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From dubois@users.sourceforge.net Fri Feb 16 11:22:02 2001 From: dubois@users.sourceforge.net (Paul F. Dubois) Date: Fri Feb 16 11:22:02 2001 Subject: documentation, was RE: [Distutils] setup.py clean questions In-Reply-To: <14988.30230.467647.418579@w221.z064000254.bwi-md.dsl.cnc.net> Message-ID: I think it is far more important at this stage to document what exists than to work on any new features. We have far more working than most of us know how to use, which is a waste. From thomas.heller@ion-tof.com Fri Feb 16 11:28:00 2001 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Fri Feb 16 11:28:00 2001 Subject: deinstall (Was: Re: [Distutils] setup.py clean questions ) References: <20010216152958.5A218371690@snelboot.oratrix.nl> <3A8D52E5.11E258CD@lemburg.com> Message-ID: <061601c09835$612e0b30$e000a8c0@thomasnotebook> > But doesn't distutils internally use Unix file name conventions ? distutils does, but glob doesn't. I think this was what Jack means. Thomas From thomas.heller@ion-tof.com Fri Feb 16 11:30:02 2001 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Fri Feb 16 11:30:02 2001 Subject: documentation, was RE: [Distutils] setup.py clean questions References: Message-ID: <062201c09835$99996730$e000a8c0@thomasnotebook> Ok, I will try to fill in some missing pieces. Thomas From jamest@math.ksu.edu Fri Feb 16 11:31:01 2001 From: jamest@math.ksu.edu (James Thompson) Date: Fri Feb 16 11:31:01 2001 Subject: documentation, was RE: [Distutils] setup.py clean questions In-Reply-To: Message-ID: I'd have to second this. I have a really tough time figuring out what setyp.py can and can't do due to the lack of docs. Digging into the source or other projects setup.py files to rediscover what others already know is very time consuming. Thanks for all the hard effort, James On Fri, 16 Feb 2001, Paul F. Dubois wrote: > I think it is far more important at this stage to document what exists than > to work on any new features. We have far more working than most of us know > how to use, which is a waste. > > > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG@python.org > http://mail.python.org/mailman/listinfo/distutils-sig > ->->->->->->->->->->->->->->->->->->---<-<-<-<-<-<-<-<-<-<-<-<-<-<-<-<-<-< James Thompson 138 Cardwell Hall Manhattan, Ks 66506 785-532-0561 Kansas State University Department of Mathematics ->->->->->->->->->->->->->->->->->->---<-<-<-<-<-<-<-<-<-<-<-<-<-<-<-<-<-< From akuchlin@mems-exchange.org Fri Feb 16 11:42:03 2001 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Fri Feb 16 11:42:03 2001 Subject: documentation, was RE: [Distutils] setup.py clean questions In-Reply-To: ; from dubois@users.sourceforge.net on Fri, Feb 16, 2001 at 08:15:01AM -0800 References: <14988.30230.467647.418579@w221.z064000254.bwi-md.dsl.cnc.net> Message-ID: <20010216114138.D5966@thrak.cnri.reston.va.us> On Fri, Feb 16, 2001 at 08:15:01AM -0800, Paul F. Dubois wrote: >I think it is far more important at this stage to document what exists than >to work on any new features. We have far more working than most of us know >how to use, which is a waste. Indeed. I'm inclined to keep saying "no" to feature requests in the 2.1 time frame, and just fix any reported bugs and work on improving the docs. Uninstallers and the like will have to be left to 2.2, possibly requiring a PEP. Feel free to continue discussing this, though; just be aware that there's no great hurry since the work won't be for 2.1. --amk From jack@oratrix.nl Fri Feb 16 11:47:11 2001 From: jack@oratrix.nl (Jack Jansen) Date: Fri Feb 16 11:47:11 2001 Subject: deinstall (Was: Re: [Distutils] setup.py clean questions ) In-Reply-To: Message by "Thomas Heller" , Fri, 16 Feb 2001 17:27:38 +0100 , <061601c09835$612e0b30$e000a8c0@thomasnotebook> Message-ID: <20010216164625.EEBA7371690@snelboot.oratrix.nl> > > But doesn't distutils internally use Unix file name conventions ? > distutils does, but glob doesn't. > I think this was what Jack means. Indeed. And, moreover, when I asked Greg whether the filenames used internally by distutils were unix filenames or URLs he answered that they are URLs. The distinction is minor but important. -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm From mal@lemburg.com Fri Feb 16 11:50:03 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri Feb 16 11:50:03 2001 Subject: [Distutils] Generic uninstall command Message-ID: <3A8D59FF.27210512@lemburg.com> Here is a different uninstall command which works by checking .get_outputs(). Rene already hinted towards this idea -- I found it while looking at the install.py source code (there's even a flag to write a list of these files somewhere...). This command should work with all install commands which adhere to the .get_outputs() logic, meaning that this API should return a list of absolute filenames of all files which the command wrote to the file system. The directory removal is a bit flaky, but works great on systems which do not allow the removal of directories which aren't empty. Unfortunately, the .get_outputs() method does not support adding directories to the list (at least the code using that method doesn't), so there doesn't seem to be a more elegant way... # # Uninstall command # class mx_uninstall(Command): description = "uninstall the package" user_options = [] def initialize_options(self): pass def finalize_options(self): pass def run(self): # Execute build self.announce('determining installation files') savevalue = self.distribution.dry_run self.distribution.dry_run = 0 self.run_command('build') # Execute install in dry-run mode self.distribution.dry_run = 1 self.run_command('install') self.distribution.dry_run = savevalue build = self.get_finalized_command('build') install = self.get_finalized_command('install') # Remove all installed files self.announce("removing files") dirs = {} filenames = install.get_outputs() for filename in filenames: if not os.path.isabs(filename): raise DistutilsError,\ 'filename "%s" from .get_output() not absolute' % \ filename if os.path.isfile(filename): self.announce("removing '%s'" % filename) if not self.dry_run: try: os.remove(filename) except OSError, details: self.warn("Could not remove file: %s" % details) dir = os.path.split(filename)[0] if not dirs.has_key(dir): dirs[dir] = 1 if os.path.splitext(filename)[1] == '.py': # Try to remove .pyc and .pyo files also try: os.remove(filename + 'c') except OSError: pass try: os.remove(filename + 'o') except OSError: pass elif os.path.isdir(filename): if not dirs.has_key(dir): dirs[filename] = 1 else: self.announce("skipping removal of '%s' (not found)" % filename) # Remove the installation directories self.announce("removing directories") dirs = dirs.keys() dirs.sort(); dirs.reverse() # sort descending for dir in dirs: self.announce("removing directory '%s'" % dir) if not self.dry_run: try: os.rmdir(dir) except OSError, details: self.warn("could not remove directory: %s" % details) -- Marc-Andre Lemburg ______________________________________________________________________ Company: http://www.egenix.com/ Consulting: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From mal@lemburg.com Fri Feb 16 11:54:01 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri Feb 16 11:54:01 2001 Subject: documentation, was RE: [Distutils] setup.py clean questions References: Message-ID: <3A8D5AA4.1D4478C2@lemburg.com> James Thompson wrote: > > I'd have to second this. I have a really tough time figuring out what > setyp.py can and can't do due to the lack of docs. Digging into the > source or other projects setup.py files to rediscover what others already > know is very time consuming. The code is very well documented, though. What's missing sometimes is an overview of how data (esp. attributes) and objects move around while executing setup.py and how the control flow really works. -- Marc-Andre Lemburg ______________________________________________________________________ Company: http://www.egenix.com/ Consulting: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From mal@lemburg.com Fri Feb 16 15:33:01 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri Feb 16 15:33:01 2001 Subject: [Distutils] Beta-release of installers for mx Extensions Message-ID: <3A8D8E3F.80692628@lemburg.com> Hi everybody, I am looking for some beta-testers of my new distutils based mx packages, the eGenix mx-Extensions. The extensions themselves are known to compile and run on practically all Python-supported platforms, but the new distutils based installation and build scheme is likely to cause some troubles which I want to move out of the way before going public. I have tested these installers on Linux and Windows without any problems, but would be interested in getting hearing about any problems they might have on the other standard Unix platforms such as AIX, HP-UX, Solaris, etc. and the more exotic ones such as MacOSX and BeOS. You can download the current snapshots from the following URLs. These RPMs are for Linux and i386 machines: http://www.lemburg.com/python/egenix-mx-base-1.0.0-1.i386.rpm http://www.lemburg.com/python/egenix-mx-base-1.0.0-1.src.rpm This archive uses the new bdist_wininst which Thomas Heller has been working on for a few weeks now: http://www.lemburg.com/python/egenix-mx-base-1.0.0.win32-py2.0.exe This archive contains the distutils based installer: http://www.lemburg.com/python/egenix-mx-base-1.0.0.zip Thanks, -- Marc-Andre Lemburg ______________________________________________________________________ Company: http://www.egenix.com/ Consulting: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From Terry@Trewitt.org Fri Feb 16 19:10:02 2001 From: Terry@Trewitt.org (Terry@Trewitt.org) Date: Fri Feb 16 19:10:02 2001 Subject: [Distutils] Re: distutils?- Borland Extensions Message-ID: (A plea for help from a Zope newbie...) I am attempting to build the elusive "_mysqldb" module required by ZMySQLDA, from , under Win98 with Borland BCC 5.5. In the post , Rene Liebscher wrote: > First convert all MSVC++ libraries you need from COFF to OMF. > coff2omf python20.lib python20_bcpp.lib I performed this step successfully > When you want to link against a library (python20), distutils checks > if there is a library with _bcpp suffix and uses it, otherwise > it tries the normal name for the library (python20.lib). This failed for me, so I replaced the entry in config.h for "python20.lib" with "python20_bcpp.lib": #pragma comment(lib,"python20_bcpp.lib") > The second point is, you have to specify that you want to use the Borland compiler. > python setup.py install build_ext --compiler=bcpp (many Thanks for this obscure tip) The error file resulting from this step is posted below Can anyone provide any enlightenment as to what I could be missing, or where I should be looking for 'LIBS.OBJ'? Are the warnings shown below usual for this module? I am neither a C or Python programmer, although I was in a former life, many years ago. I found and loaded BCC just for this project. On Win98, I have loaded: -- Be-Open binary Win32 distribution/installation of Python 2.0 , BeOpen-Python-2.0.exe, with include files from BeOpen-Python-2.0.tar.gz -- MySQL-python-0.3.3.tar.gz -- ZMySQLDA-2.0.4.tar.gz (this is the package I wish to use, with LoginManager in Zope) (Next question...) My ultimate goal in using LoginManager is to enable members of a ski club to: -- Apply for membership -- Update their own information, including access passwords -- Sign up for trips and events -- Post articles, chat and view the membership list, after authentication -- Allow for management of user information and the remainder of the site by a small core of administrators Are there limitations in LoginManager that would limit its use in performing the authentication and session management required by the above? Thanks in advance.... Terry Trewitt Terry@Trewitt.org ---------------------------------------------------------------------- The creation of these 2 vital files was suggested by Gordon Williams (Many Thanks!) ------------------------------------------------ C:\Borland\bcc32\bin\bcc32.cfg: ------------------------------------------------ -w -I"c:\BORLAND\BCC55\INCLUDE;.\;D:\PROGRAM FILES\LSS-ZOPE\bin\PYTHON20\INCLUDE;d:\nusphere\mysql\include" -L"c:\BORLAND\BCC55\LIB;c:\Borland\Bcc55\lib\psdk;D:\PROGRAM FILES\LSS-ZOPE\bin\PYTHON20\LIBS;d:\nusphere\mysql\lib\opt" ------------------------------------------------ C:\Borland\bcc32\bin\ilink32.cfg: ------------------------------------------------ -x -L"C:\BORLAND\BCC55\LIB;C:\Borland\bcc55\lib\psdk;D:\PROGRAM FILES\LSS-ZOPE\bin\PYTHON20\LIBS" ------------------------------------------------ python setup.py install build_ext --compiler=bcpp > build.log ------------------------------------------------ _mysqlmodule.c: Warning W8060 _mysqlmodule.c 81: Possibly incorrect assignment in function _mysql_Exception Warning W8060 _mysqlmodule.c 381: Possibly incorrect assignment in function _mysql_ResultObject_New Warning W8060 _mysqlmodule.c 393: Possibly incorrect assignment in function _mysql_ResultObject_New Warning W8060 _mysqlmodule.c 455: Possibly incorrect assignment in function _mysql_connect Warning W8057 _mysqlmodule.c 489: Parameter 'self' is never used in function _mysql_connect Warning W8004 _mysqlmodule.c 461: 'conn' is assigned a value that is never used in function _mysql_connect Warning W8057 _mysqlmodule.c 524: Parameter 'self' is never used in function _mysql_debug Warning W8060 _mysqlmodule.c 627: Possibly incorrect assignment in function _escape_item Warning W8057 _mysqlmodule.c 663: Parameter 'self' is never used in function _mysql_escape Warning W8060 _mysqlmodule.c 680: Possibly incorrect assignment in function _mysql_escape_sequence Warning W8057 _mysqlmodule.c 691: Parameter 'self' is never used in function _mysql_escape_sequence Warning W8060 _mysqlmodule.c 707: Possibly incorrect assignment in function _mysql_escape_dict Warning W8057 _mysqlmodule.c 717: Parameter 'self' is never used in function _mysql_escape_dict Warning W8060 _mysqlmodule.c 731: Possibly incorrect assignment in function _mysql_ResultObject_describe Warning W8060 _mysqlmodule.c 762: Possibly incorrect assignment in function _mysql_ResultObject_field_flags Warning W8060 _mysqlmodule.c 765: Possibly incorrect assignment in function _mysql_ResultObject_field_flags Warning W8060 _mysqlmodule.c 809: Possibly incorrect assignment in function _mysql_row_to_tuple Warning W8060 _mysqlmodule.c 835: Possibly incorrect assignment in function _mysql_row_to_dict Warning W8060 _mysqlmodule.c 874: Possibly incorrect assignment in function _mysql_row_to_dict_old Warning W8012 _mysqlmodule.c 915: Comparing signed and unsigned values in function _mysql__fetch_row Warning W8008 _mysqlmodule.c 962: Condition is always false in function _mysql_ResultObject_fetch_row Warning W8060 _mysqlmodule.c 968: Possibly incorrect assignment in function _mysql_ResultObject_fetch_row Warning W8060 _mysqlmodule.c 975: Possibly incorrect assignment in function _mysql_ResultObject_fetch_row Warning W8060 _mysqlmodule.c 986: Possibly incorrect assignment in function _mysql_ResultObject_fetch_row Warning W8004 _mysqlmodule.c 957: 'r' is assigned a value that is never used in function _mysql_ResultObject_fetch_row Warning W8057 _mysqlmodule.c 1041: Parameter 'self' is never used in function _mysql_get_client_info Warning W8057 _mysqlmodule.c 1051: Parameter 'self' is never used in function _mysql_ConnectionObject_commit Warning W8060 _mysqlmodule.c 1625: Possibly incorrect assignment in function _mysql_Constant_class Warning W8060 _mysqlmodule.c 1627: Possibly incorrect assignment in function _mysql_Constant_class Warning W8060 _mysqlmodule.c 1630: Possibly incorrect assignment in function _mysql_Constant_class Warning W8060 _mysqlmodule.c 1687: Possibly incorrect assignment in function init_mysql Warning W8060 _mysqlmodule.c 1690: Possibly incorrect assignment in function init_mysql Warning W8060 _mysqlmodule.c 1693: Possibly incorrect assignment in function init_mysql Warning W8060 _mysqlmodule.c 1696: Possibly incorrect assignment in function init_mysql Warning W8060 _mysqlmodule.c 1699: Possibly incorrect assignment in function init_mysql Warning W8060 _mysqlmodule.c 1702: Possibly incorrect assignment in function init_mysql Warning W8060 _mysqlmodule.c 1706: Possibly incorrect assignment in function init_mysql Warning W8060 _mysqlmodule.c 1710: Possibly incorrect assignment in function init_mysql Warning W8060 _mysqlmodule.c 1714: Possibly incorrect assignment in function init_mysql Warning W8060 _mysqlmodule.c 1718: Possibly incorrect assignment in function init_mysql Warning W8060 _mysqlmodule.c 1732: Possibly incorrect assignment in function init_mysql Fatal: Unable to open file 'LIBS.OBJ' running install running build running build_py copying MySQLdb.py -> build\lib.win32-2.0 copying CompatMysqldb.py -> build\lib.win32-2.0 running build_ext building '_mysql' extension C:\BORLAND\BCC55\BIN\bcc32.exe -c /tWM /O2 /q /g0 -Id:\nusphere\mysql\include "-ID:\PROGRAM FILES\LSS-ZOPE\BIN\PYTHON20\Include" -obuild\temp.win32-2.0\Release\_mysqlmodule.obj _mysqlmodule.c writing build\temp.win32-2.0\Release\_mysql.def C:\BORLAND\BCC55\BIN\ilink32.exe /Tpd /Gn /q /x /Ld:\nusphere\mysql\lib\opt "/LD:\PROGRAM FILES\LSS-ZOPE\BIN\PYTHON20\libs" /L. c0d32 build\temp.win32-2.0\Release\_mysqlmodule.obj d:\nusphere\mysql\lib\opt\mysqlclient.lib , build\lib.win32-2.0\_mysql.pyd ,, d:\nusphere\mysql\lib\opt\mysqlclient.lib d:\nusphere\mysql\lib\opt\zlib.lib msvcrt libcmt wsock32 advapi32 "D:\PROGRAM FILES\LSS-ZOPE\BIN\PYTHON20\libs\python20_bcpp.lib" import32 cw32mt , build\temp.win32-2.0\Release\_mysql.def , From paulp@ActiveState.com Fri Feb 16 20:18:01 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Fri Feb 16 20:18:01 2001 Subject: [Distutils] Generic uninstall command References: <3A8D59FF.27210512@lemburg.com> Message-ID: <3A8DD1A0.F2838A1D@ActiveState.com> "M.-A. Lemburg" wrote: > > ... > > This command should work with all install commands which adhere to > the .get_outputs() logic, meaning that this API should return > a list of absolute filenames of all files which the command wrote > to the file system. Am I wrong to be a little bit paranoid about files used by more than one distribution? Paul Prescod From akuchlin@mems-exchange.org Fri Feb 16 23:53:00 2001 From: akuchlin@mems-exchange.org (A.M. Kuchling) Date: Fri Feb 16 23:53:00 2001 Subject: [Distutils] Runtime library directories Message-ID: <200102170506.AAA05138@mira.erols.com> Something I noticed this evening: unixccompiler.py uses -R for runtime library directives. That's good syntax for ld, but is it good syntax for C compilers? GCC requires -Wl,-R instead to pass it to the linker; is this standard for Unix C compilers? I suspect this patch below would be too simple to fix the problem, so how to fix this? --amk Index: unixccompiler.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/distutils/unixccompiler.py,v retrieving revision 1.32 diff -u -p -r1.32 unixccompiler.py --- unixccompiler.py 2000/09/27 02:08:14 1.32 +++ unixccompiler.py 2001/02/17 04:51:37 @@ -247,7 +247,7 @@ class UnixCCompiler (CCompiler): return "-L" + dir def runtime_library_dir_option (self, dir): - return "-R" + dir + return "-Wl,-R" + dir def library_option (self, lib): return "-l" + lib From mal@lemburg.com Sat Feb 17 04:39:01 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Sat Feb 17 04:39:01 2001 Subject: [Distutils] Generic uninstall command References: <3A8D59FF.27210512@lemburg.com> <3A8DD1A0.F2838A1D@ActiveState.com> Message-ID: <3A8E465F.866E400C@lemburg.com> Paul Prescod wrote: > > "M.-A. Lemburg" wrote: > > > > ... > > > > This command should work with all install commands which adhere to > > the .get_outputs() logic, meaning that this API should return > > a list of absolute filenames of all files which the command wrote > > to the file system. > > Am I wrong to be a little bit paranoid about files used by more than one > distribution? The uninstall command will simply reverse the process of installing a package. It does not try to manage file ownership or dependencies. This would be a whole different subject and a very complex one too. I'm using the above code in the installer for the eGenix mx-Extensions and these don't share any files. -- Marc-Andre Lemburg ______________________________________________________________________ Company: http://www.egenix.com/ Consulting: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From mal@lemburg.com Sat Feb 17 05:24:02 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Sat Feb 17 05:24:02 2001 Subject: [Distutils] Runtime library directories References: <200102170506.AAA05138@mira.erols.com> Message-ID: <3A8E5102.BE973D42@lemburg.com> "A.M. Kuchling" wrote: > > Something I noticed this evening: unixccompiler.py uses -R for > runtime library directives. That's good syntax for ld, but is it good > syntax for C compilers? GCC requires -Wl,-R instead to pass it > to the linker; is this standard for Unix C compilers? I suspect this > patch below would be too simple to fix the problem, so how to fix > this? Are you sure that the ld -R flag is really what you want here ? The docs say: -R filename Read symbol names and their addresses from file­ name, but do not relocate it or include it in the output. This allows your output file to refer sym­ bolically to absolute locations of memory defined in other programs. I think that the -rpath ld option is what you are really looking for: -rpath directory Add a directory to the runtime library search path. This is used when linking an ELF executable with shared objects. All -rpath arguments are concate­ nated and passed to the runtime linker, which uses them to locate shared objects at runtime. The -rpath option is also used when locating shared ob­ jects which are needed by shared objects explicitly included in the link; see the description of the -rpath-link option. If -rpath is not used when linking an ELF executable, the contents of the en­ vironment variable LD_RUN_PATH will be used if it is defined. The -rpath option may also be used on SunOS. By default, on SunOS, the linker will form a runtime search patch out of all the -L options it is given. If a -rpath option is used, the runtime search path will be formed exclusively using the -rpath op­ tions, ignoring the -L options. This can be useful when using gcc, which adds many -L options which may be on NFS mounted filesystems. In any case, I'd say you switch on the compiler name and leave the -R option in place for all non-gcc compilers until more testing has been done in this area. > --amk > > Index: unixccompiler.py > =================================================================== > RCS file: /cvsroot/python/python/dist/src/Lib/distutils/unixccompiler.py,v > retrieving revision 1.32 > diff -u -p -r1.32 unixccompiler.py > --- unixccompiler.py 2000/09/27 02:08:14 1.32 > +++ unixccompiler.py 2001/02/17 04:51:37 > @@ -247,7 +247,7 @@ class UnixCCompiler (CCompiler): > return "-L" + dir > > def runtime_library_dir_option (self, dir): > - return "-R" + dir > + return "-Wl,-R" + dir > > def library_option (self, lib): > return "-l" + lib > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG@python.org > http://mail.python.org/mailman/listinfo/distutils-sig -- Marc-Andre Lemburg ______________________________________________________________________ Company: http://www.egenix.com/ Consulting: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From mal@lemburg.com Sat Feb 17 09:17:00 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Sat Feb 17 09:17:00 2001 Subject: [Distutils] Adding --compiler pass-through option to install Message-ID: <3A8E879F.B555BEE6@lemburg.com> Wouldn't it make sense to add a --compiler pass-through option to the install command ? That way, users having non-default compiler setups will be able to install distutils packages using one command only: python setup.py install --compiler mycompiler instead of having to run python setup.py build --compiler mycompiler python setup.py install -- Marc-Andre Lemburg ______________________________________________________________________ Company: http://www.egenix.com/ Consulting: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From Alexandre.Fayolle@logilab.fr Sat Feb 17 13:55:01 2001 From: Alexandre.Fayolle@logilab.fr (Alexandre Fayolle) Date: Sat Feb 17 13:55:01 2001 Subject: [Distutils] Re: [4suite] Re: Compiling 4Suite 0.10.2beta on Windows In-Reply-To: <200102171818.f1HIIv309798@mira.informatik.hu-berlin.de> Message-ID: On Sat, 17 Feb 2001, Martin v. Loewis wrote: > > OK, let's phrase it differently. How do you build the 4Suite windows > > installer when time comes for a release ? > > Maybe I'm confusing something, but for PyXML, I build the installer > with > > python setup.py bdist_wininst OK, I misworded my question. Actually, this may be a distutils-sig question, so I crosspost to the distutils ML. I tried running bdist_wininst on my linux box, and this failed, because "distribution contains extensions and/or C libraries; must be compiled on a Windows 32 platform". So I switched to a Win32 box, and ran bdist_wininst, and it failed too because I could non find CL.EXE. What is CL.EXE? I suppose it is a compiler, so it is probably the MS compiler from Visual Studio. Can anyone confirm this? Is it possible to use mingw's gcc or cygwin's to build python extensions on windows? TIA Alexandre Fayolle -- http://www.logilab.com Narval is the first software agent available as free software (GPL). LOGILAB, Paris (France). From chapmanb@arches.uga.edu Sat Feb 17 15:35:01 2001 From: chapmanb@arches.uga.edu (Brad Chapman) Date: Sat Feb 17 15:35:01 2001 Subject: [Distutils] Re: [4suite] Re: Compiling 4Suite 0.10.2beta on Windows In-Reply-To: References: <200102171818.f1HIIv309798@mira.informatik.hu-berlin.de> Message-ID: <14990.57410.690077.528746@taxus.athen1.ga.home.com> [Only posted to distutils-sig] Hi Alexandre; > So I switched to a Win32 box, and ran bdist_wininst, and it failed too > because I could non find CL.EXE. What is CL.EXE? I suppose it is a > compiler, so it is probably the MS compiler from Visual Studio. Can anyone > confirm this? Is it possible to use mingw's gcc or cygwin's to build > python extensions on windows? I also ran into similar problems when I was trying to learn to build something on Windows, since I normally work on Windows. I don't know about cygwin or mingw, but I compiled my stuff using Borland's free compiler. I stuck instructions for how I did this in the Tutorial for a project that I work on: http://www.bioinformatics.org/bradstuff/bp/tut/Tutorial.html Section 1.3.2 has the information I know about compiling it. After compiling with Borland, then you just need to do the 'python setup.py bdist_winist' Hope this helps, Brad From martin@loewis.home.cs.tu-berlin.de Sat Feb 17 19:23:02 2001 From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis) Date: Sat Feb 17 19:23:02 2001 Subject: [Distutils] Re: [4suite] Re: Compiling 4Suite 0.10.2beta on Windows In-Reply-To: (message from Alexandre Fayolle on Sat, 17 Feb 2001 19:57:11 +0100 (CET)) References: Message-ID: <200102180019.f1I0JjQ10902@mira.informatik.hu-berlin.de> > I tried running bdist_wininst on my linux box, and this failed, > because "distribution contains extensions and/or C libraries; must > be compiled on a Windows 32 platform". This is expected. Distutils *could* look for a cross-compiler, but it currently doesn't, and likely you don't have one, either. > So I switched to a Win32 box, and ran bdist_wininst, and it failed > too because I could non find CL.EXE. What is CL.EXE? I suppose it is > a compiler, so it is probably the MS compiler from Visual > Studio. Can anyone confirm this? That is correct. If you have MSVC installed, it did ask you whether you wanted to register the path for its binaries. If you did that, distutils should find cl (which is the compiler proper, I believe) right away. If you didn't, you'll have to invoke vcvars32.bat first (or is it vc32vars?). > Is it possible to use mingw's gcc or cygwin's to build python > extensions on windows? I'm not certain what the answer to that question is. Using cygwin is definitely a bad idea; if anything, mingw is the right approach. I think Python 2.0 supports building with cygwin. Building extensions involves a number of issues: 1. I'm not sure whether GNU ld supports MSVC import libraries. That may also depend on the MSVC version that was used to create the import libraries, and the version of GNU ld that you have. Looking at distutils.cygwincompiler, it appears that you have to build your own import library first, or that you have to build Python with gcc. It would probably be good if some volunteer could provide a mingw import library of python20.dll, and if python 2.1 could provide such an import library as a battery. 2. I'm not sure whether you can talk distutils into using mingw. In theory, passing --compiler=mingw32 to the build command should work; see setup.py build --help-compiler for details. Regards, Martin P.S. Since you originally wanted to know how 4Suite is build: most likely using MSVC 6, since that is what everybody else uses. From phrxy@csv.warwick.ac.uk Sat Feb 17 22:38:59 2001 From: phrxy@csv.warwick.ac.uk (John J. Lee) Date: Sat Feb 17 22:38:59 2001 Subject: [Distutils] Adding --compiler pass-through option to install In-Reply-To: <3A8E879F.B555BEE6@lemburg.com> Message-ID: On Sat, 17 Feb 2001, M.-A. Lemburg wrote: > Wouldn't it make sense to add a --compiler pass-through option > to the install command ? > > That way, users having non-default compiler setups will be able > to install distutils packages using one command only: > > python setup.py install --compiler mycompiler > > instead of having to run > > python setup.py build --compiler mycompiler > python setup.py install > [...] I agree, but in fact the latter didn't work for me when compiling FORTRAN (I added a compiler 'unixf' that was just a trivial subclass of 'unixc'). It would compile with python setup.py build --compiler unixf but then would try to rebuild it again on python setup.py install I didn't try to find out why because I thought adding a compiler was probably the wrong way to do what I was doing anyway, but does anyone know exactly why this happens? John From Jason.Tishler@dothill.com Sat Feb 17 22:41:03 2001 From: Jason.Tishler@dothill.com (Jason Tishler) Date: Sat Feb 17 22:41:03 2001 Subject: [Distutils] Re: [4suite] Re: Compiling 4Suite 0.10.2beta on Windows In-Reply-To: <200102180019.f1I0JjQ10902@mira.informatik.hu-berlin.de>; from martin@loewis.home.cs.tu-berlin.de on Sun, Feb 18, 2001 at 01:19:45AM +0100 References: <200102180019.f1I0JjQ10902@mira.informatik.hu-berlin.de> Message-ID: <20010217224412.A226@dothill.com> On Sun, Feb 18, 2001 at 01:19:45AM +0100, Martin v. Loewis wrote: > > Is it possible to use mingw's gcc or cygwin's to build python > > extensions on windows? > > I'm not certain what the answer to that question is. Using cygwin is > definitely a bad idea; if anything, mingw is the right approach. I wouldn't say that using Cygwin is a bad idea -- I'd rather state that if you do so then your extension is dependent on Cygwin's POSIX emulation layer (i.e., cygwin1.dll) and that you need Cygwin Python to execute it. Hence, if you are only interested in Win32 Python, then mingw is the better choice. However, Python 2.1a2 builds OOTB under Cygwin and supports both the traditional (i.e. Misc/Makefile.pre.in) and distutils methods of building shared extensions. If interested, see the following: http://www.cygwin.com/ml/cygwin-apps/2001-02/msg00004.html http://www.cygwin.com/ml/cygwin-apps/2001-01/msg00019.html Jason -- Jason Tishler Director, Software Engineering Phone: +1 (732) 264-8770 x235 Dot Hill Systems Corp. Fax: +1 (732) 264-8798 82 Bethany Road, Suite 7 Email: Jason.Tishler@dothill.com Hazlet, NJ 07730 USA WWW: http://www.dothill.com From martin@loewis.home.cs.tu-berlin.de Sun Feb 18 05:21:00 2001 From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis) Date: Sun Feb 18 05:21:00 2001 Subject: [Distutils] Re: [4suite] Re: Compiling 4Suite 0.10.2beta on Windows In-Reply-To: <20010217224412.A226@dothill.com> (message from Jason Tishler on Sat, 17 Feb 2001 22:44:12 -0500) References: <200102180019.f1I0JjQ10902@mira.informatik.hu-berlin.de> <20010217224412.A226@dothill.com> Message-ID: <200102180955.f1I9t3700868@mira.informatik.hu-berlin.de> > I wouldn't say that using Cygwin is a bad idea -- I'd rather state that > if you do so then your extension is dependent on Cygwin's POSIX emulation > layer (i.e., cygwin1.dll) and that you need Cygwin Python to execute it. That's what I meant - if you have an MSVC-built Python, than a cygwin-built extension probably won't work, will it? > However, Python 2.1a2 builds OOTB under Cygwin and supports both the > traditional (i.e. Misc/Makefile.pre.in) and distutils methods of > building shared extensions. That's good to hear. So what about using Cygwin to build distutils-made extensions when the Python installation was built with MSVC? Regards, Martin From akuchlin@mems-exchange.org Sun Feb 18 17:58:02 2001 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Sun Feb 18 17:58:02 2001 Subject: [Distutils] Adding --compiler pass-through option to install In-Reply-To: <3A8E879F.B555BEE6@lemburg.com>; from mal@lemburg.com on Sat, Feb 17, 2001 at 03:15:59PM +0100 References: <3A8E879F.B555BEE6@lemburg.com> Message-ID: <20010218175729.A26452@ute.cnri.reston.va.us> On Sat, Feb 17, 2001 at 03:15:59PM +0100, M.-A. Lemburg wrote: >Wouldn't it make sense to add a --compiler pass-through option >to the install command ? IMHO that leads down the slippery slope of having to pass everything through; --libraries and --debug and --define &c. would also all have to be passed through, and applied to the proper command. If you need special arguments, run the passes separately. --amk From R.Liebscher@gmx.de Mon Feb 19 06:59:01 2001 From: R.Liebscher@gmx.de (Rene Liebscher) Date: Mon Feb 19 06:59:01 2001 Subject: [Distutils] Adding --compiler pass-through option to install References: <3A8E879F.B555BEE6@lemburg.com> <20010218175729.A26452@ute.cnri.reston.va.us> Message-ID: <3A910A37.B6FC6F2D@gmx.de> Andrew Kuchling wrote: > > On Sat, Feb 17, 2001 at 03:15:59PM +0100, M.-A. Lemburg wrote: > >Wouldn't it make sense to add a --compiler pass-through option > >to the install command ? > > IMHO that leads down the slippery slope of having to pass everything > through; --libraries and --debug and --define &c. would also all have > to be passed through, and applied to the proper command. If you need > special arguments, run the passes separately. > If you have to specify it for every package you compile, because you want to use a particular compiler, then you should write a personal configuration file as described in http://www.python.org/sigs/distutils-sig/doc/inst/sec:config-filenames.html Kind regards Rene Liebscher From R.Liebscher@gmx.de Mon Feb 19 07:04:02 2001 From: R.Liebscher@gmx.de (Rene Liebscher) Date: Mon Feb 19 07:04:02 2001 Subject: [Distutils] Re: [4suite] Re: Compiling 4Suite 0.10.2beta on Windows References: Message-ID: <3A910B3B.965DCA07@gmx.de> Alexandre Fayolle wrote: > > On Sat, 17 Feb 2001, Martin v. Loewis wrote: > > > > OK, let's phrase it differently. How do you build the 4Suite windows > > > installer when time comes for a release ? > > > > Maybe I'm confusing something, but for PyXML, I build the installer > > with > > > > python setup.py bdist_wininst > > OK, I misworded my question. Actually, this may be a distutils-sig > question, so I crosspost to the distutils ML. > > I tried running bdist_wininst on my linux box, and this failed, because > "distribution contains extensions and/or C libraries; must be compiled on > a Windows 32 platform". > > So I switched to a Win32 box, and ran bdist_wininst, and it failed too > because I could non find CL.EXE. What is CL.EXE? I suppose it is a > compiler, so it is probably the MS compiler from Visual Studio. Can anyone > confirm this? Is it possible to use mingw's gcc or cygwin's to build > python extensions on windows? > You can build extensions with Borland C++ by specifying --compiler=bcpp for the build or build_ext command (bdist_wininst calls intern build and install) (You probably should create a personal configuration for your distutils see http://www.python.org/sigs/distutils-sig/doc/inst/sec:config-filenames.html ) For MinGW and Cygwin in MinGW-mode it is --compiler=mingw32 For Cygwin in Cygwin-mode it is --compiler=cygwin With --compiler=mingw32 compiled extensions run without problems on an MSVC built Python. Some simpler examples also worked when compiled with --compiler=cygwin (I never tested it thorougly because I prefer not to need an additional dll.) Kind regards Rene Liebscher From R.Liebscher@gmx.de Mon Feb 19 07:27:59 2001 From: R.Liebscher@gmx.de (Rene Liebscher) Date: Mon Feb 19 07:27:59 2001 Subject: [Distutils] Re: distutils?- Borland Extensions References: Message-ID: <3A91111E.AA567198@gmx.de> Terry@trewitt.org wrote: > > (A plea for help from a Zope newbie...) > > I am attempting to build the elusive "_mysqldb" module required by > ZMySQLDA, from > , under > Win98 with Borland BCC 5.5. > > In the post > , > Rene Liebscher wrote: > > > First convert all MSVC++ libraries you need from COFF to OMF. > > coff2omf python20.lib python20_bcpp.lib > > I performed this step successfully > > > When you want to link against a library (python20), distutils checks > > if there is a library with _bcpp suffix and uses it, otherwise > > it tries the normal name for the library (python20.lib). > > This failed for me, so I replaced the entry in config.h for "python20.lib" > with "python20_bcpp.lib": > #pragma comment(lib,"python20_bcpp.lib") This makes no sense because it is enclosed in an #ifdef _MSC_VER (python20_bcpp.lib was found!!! see your command line "D:\PROGRAM FILES\LSS-ZOPE\BIN\PYTHON20\libs\python20_bcpp.lib" ) > > The second point is, you have to specify that you want to use the > Borland compiler. > > python setup.py install build_ext --compiler=bcpp > > (many Thanks for this obscure tip) > The error file resulting from this step is posted below > > Can anyone provide any enlightenment as to what I could be missing, or > where I should be looking for 'LIBS.OBJ'? Are the warnings shown below > usual for this module? I am neither a C or Python programmer, although I > was in a former life, many years ago. I found and loaded BCC just for > this project. (The warnings are not a problem I think, because Borland C++ brings more warnings than most other compilers.) I think the problem (LIBS.OBJ) is the following part of the command line C:\BORLAND\BCC55\BIN\ilink32.exe /Tpd /Gn /q /x /Ld:\nusphere\mysql\lib\opt ==> "/LD:\PROGRAM FILES\LSS-ZOPE\BIN\PYTHON20\libs" <== /L. c0d32 build\temp.win32-2.0\Release\_mysqlmodule.obj d:\nusphere\mysql\lib\opt\mysqlclient.lib , build\lib.win32-2.0\_mysql.pyd ,, d:\nusphere\mysql\lib\opt\mysqlclient.lib d:\nusphere\mysql\lib\opt\zlib.lib msvcrt libcmt wsock32 advapi32 "D:\PROGRAM FILES\LSS-ZOPE\BIN\PYTHON20\libs\python20_bcpp.lib" import32 cw32mt , build\temp.win32-2.0\Release\_mysql.def , Because of the quoting ilink might take this as an object filename. (If it is not this, it might be the similar path name in the ilink32.cfg file.) It might be help to patch distutils/bcppcompiler.py as follows ----------------------------------------------------------------------- --- bcppcompiler.py Mon Feb 12 10:46:21 2001 +++ bcppcompiler_new.py Mon Feb 19 13:18:51 2001 @@ -257,7 +257,7 @@ for l in library_dirs: - ld_args.append("/L%s" % os.path.normpath(l)) + ld_args.extend(["/L","%s" % os.path.normpath(l)]) ld_args.append("/L.") # we sometimes use relative paths # list of object files ------------------------------------------------------------------------ I don't know if it helps, because I currently have no (Windows,Borland C++) computer available to test it. If it works send me a note, so we can make it an offical patch for distutils. Kind regards Rene Liebscher > ... > > ---------------------------------------------------------------------- > The creation of these 2 vital files was suggested by Gordon Williams > (Many Thanks!) > ------------------------------------------------ > C:\Borland\bcc32\bin\bcc32.cfg: > ------------------------------------------------ > -w > -I"c:\BORLAND\BCC55\INCLUDE;.\;D:\PROGRAM > FILES\LSS-ZOPE\bin\PYTHON20\INCLUDE;d:\nusphere\mysql\include" > -L"c:\BORLAND\BCC55\LIB;c:\Borland\Bcc55\lib\psdk;D:\PROGRAM > FILES\LSS-ZOPE\bin\PYTHON20\LIBS;d:\nusphere\mysql\lib\opt" > > ------------------------------------------------ > C:\Borland\bcc32\bin\ilink32.cfg: > ------------------------------------------------ > -x > -L"C:\BORLAND\BCC55\LIB;C:\Borland\bcc55\lib\psdk;D:\PROGRAM > FILES\LSS-ZOPE\bin\PYTHON20\LIBS" > > ------------------------------------------------ > python setup.py install build_ext --compiler=bcpp > build.log > ------------------------------------------------ > _mysqlmodule.c: > Warning W8060 _mysqlmodule.c 81: Possibly incorrect assignment in function > _mysql_Exception > ... > Warning W8060 _mysqlmodule.c 1732: Possibly incorrect assignment in > function init_mysql > Fatal: Unable to open file 'LIBS.OBJ' > running install > running build > running build_py > copying MySQLdb.py -> build\lib.win32-2.0 > copying CompatMysqldb.py -> build\lib.win32-2.0 > running build_ext > building '_mysql' extension > C:\BORLAND\BCC55\BIN\bcc32.exe -c /tWM /O2 /q /g0 > -Id:\nusphere\mysql\include "-ID:\PROGRAM > FILES\LSS-ZOPE\BIN\PYTHON20\Include" > -obuild\temp.win32-2.0\Release\_mysqlmodule.obj _mysqlmodule.c > writing build\temp.win32-2.0\Release\_mysql.def > C:\BORLAND\BCC55\BIN\ilink32.exe /Tpd /Gn /q /x > /Ld:\nusphere\mysql\lib\opt "/LD:\PROGRAM > FILES\LSS-ZOPE\BIN\PYTHON20\libs" /L. c0d32 > build\temp.win32-2.0\Release\_mysqlmodule.obj > d:\nusphere\mysql\lib\opt\mysqlclient.lib , build\lib.win32-2.0\_mysql.pyd > ,, d:\nusphere\mysql\lib\opt\mysqlclient.lib > d:\nusphere\mysql\lib\opt\zlib.lib msvcrt libcmt wsock32 advapi32 > "D:\PROGRAM FILES\LSS-ZOPE\BIN\PYTHON20\libs\python20_bcpp.lib" import32 > cw32mt , build\temp.win32-2.0\Release\_mysql.def , From thomas.heller@ion-tof.com Mon Feb 19 08:15:59 2001 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Mon Feb 19 08:15:59 2001 Subject: [Distutils] Documentation Message-ID: <004301c09a75$f7459ec0$e000a8c0@thomasnotebook> Here is what I've hacked together over some spare time at the weekend. I would appreciate feedback, as I've told before I'm neither a great author, nor native english speaker, nor anything else. Mostly I've completed (and written) docs for bdist_wininst (where I'm an expertert for), also I've started some stuff for the data_files and scripts option. Should I continue in this way? Would others like to jump in? Thomas Index: dist.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/dist/dist.tex,v retrieving revision 1.29 diff -c -r1.29 dist.tex *** dist.tex 2000/11/22 16:06:16 1.29 --- dist.tex 2001/02/19 13:07:06 *************** *** 53,59 **** Using the Distutils is quite simple, both for module developers and for users/administrators installing third-party modules. As a developer, ! your responsibilites (apart from writing solid, well-documented and well-tested code, of course!) are: \begin{itemize} \item write a setup script (\file{setup.py} by convention) --- 53,59 ---- Using the Distutils is quite simple, both for module developers and for users/administrators installing third-party modules. As a developer, ! your responsibilities (apart from writing solid, well-documented and well-tested code, of course!) are: \begin{itemize} \item write a setup script (\file{setup.py} by convention) *************** *** 148,166 **** will create an executable installer, \file{Foo-1.0.win32.exe}, in the current directory. ! \XXX{not implemented yet} ! (Another way to create executable installers for Windows is with the ! \command{bdist\_wise} command, which uses Wise---the commercial ! installer-generator used to create Python's own installer---to create ! the installer. Wise-based installers are more appropriate for large, ! industrial-strength applications that need the full capabilities of a ! ``real'' installer. \command{bdist\_wininst} creates a self-extracting ! zip file with a minimal user interface, which is enough for small- to ! medium-sized module collections. You'll need to have version XXX of ! Wise installed on your system for the \command{bdist\_wise} command to ! work; it's available from \url{http://foo/bar/baz}.) ! ! Currently (Distutils 0.9.2), the are only other useful built distribution format is RPM, implemented by the \command{bdist\_rpm} command. For example, the following command will create an RPM file called \file{Foo-1.0.noarch.rpm}: --- 148,154 ---- will create an executable installer, \file{Foo-1.0.win32.exe}, in the current directory. ! Currently (Distutils 0.9.2), the only other useful built distribution format is RPM, implemented by the \command{bdist\_rpm} command. For example, the following command will create an RPM file called \file{Foo-1.0.noarch.rpm}: *************** *** 192,198 **** single \file{.py} file (and possibly associated \file{.pyc} and/or \file{.pyo} files). Sometimes referred to as a ``pure module.'' \item[extension module] a module written in the low-level language of ! the Python implemention: C/C++ for Python, Java for JPython. Typically contained in a single dynamically loadable pre-compiled file, e.g. a shared object (\file{.so}) file for Python extensions on \UNIX, a DLL (given the \file{.pyd} extension) for Python extensions --- 180,186 ---- single \file{.py} file (and possibly associated \file{.pyc} and/or \file{.pyo} files). Sometimes referred to as a ``pure module.'' \item[extension module] a module written in the low-level language of ! the Python implementation: C/C++ for Python, Java for JPython. Typically contained in a single dynamically loadable pre-compiled file, e.g. a shared object (\file{.so}) file for Python extensions on \UNIX, a DLL (given the \file{.pyd} extension) for Python extensions *************** *** 240,246 **** \section{Writing the Setup Script} \label{setup-script} ! The setup script is the centre of all activity in building, distributing, and installing modules using the Distutils. The main purpose of the setup script is to describe your module distribution to the Distutils, so that the various commands that operate on your modules --- 228,234 ---- \section{Writing the Setup Script} \label{setup-script} ! The setup script is the center of all activity in building, distributing, and installing modules using the Distutils. The main purpose of the setup script is to describe your module distribution to the Distutils, so that the various commands that operate on your modules *************** *** 290,295 **** --- 278,291 ---- mind that the \emph{absence} of a leading slash indicates a relative path, the opposite of the MacOS convention with colons). + This, of course, only applies to pathnames given to Distutils functions. + If you, for example, use standard python functions such as glob.glob + or os.listdir to specify files, you should be careful to write portable + code instead of hardcoding path separators: + \begin{verbatim} + glob.glob(os.path.join('mydir', 'subdir', '*.html')) + os.listdir(os.path.join('mydir', 'subdir')) + \end{verbatim} \subsection{Listing whole packages} \label{listing-packages} *************** *** 447,454 **** On some platforms, you can include non-source files that are processed by the compiler and included in your extension. Currently, this just ! means Windows resource files for Visual C++. \XXX{get more detail on ! this feature from Thomas Heller!} \subsubsection{Preprocessor options} --- 443,451 ---- On some platforms, you can include non-source files that are processed by the compiler and included in your extension. Currently, this just ! means Windows message text (\file{.mc}) files and resource definition ! (\file{.rc}) files for Visual C++. These will be compiled to binary resource ! (\file{.res}) files and linked into the executable. \subsubsection{Preprocessor options} *************** *** 549,560 **** \end{verbatim} (Again, this sort of non-portable construct should be avoided if you intend to distribute your code.) ! \XXX{still undocumented: extra\_objects, extra\_compile\_args, ! extra\_link\_args, export\_symbols---none of which are frequently ! needed, some of which might be completely unnecessary!} \section{Writing the Setup Configuration File} \label{setup-config} --- 546,611 ---- \end{verbatim} (Again, this sort of non-portable construct should be avoided if you intend to distribute your code.) + + \XXX{THe Should mention clib libraries here or somewhere else!} + + \subsubsection{Other options} + + There are still some other options which can be used to handle special + cases. + + The \option{extra\_objects} option is a list of object files to be passed + to the linker. These files must not have extensions, the default extension + for the compiler is used. ! \option{extra\_compile\_args} and \option{extra\_link\_args} can be used ! to specify additional command line options for the compiler resp. ! the linker command line. + \option{export\_symbols} is only useful on windows, it can contain a list + of symbols (funtions or variables) to be exported. This option + is not needed when building compiled extensions: the \code{initmodule} + function will automatically be added to the exported symbols list + by Distutils. + \subsection{Listing scripts} + So far we have been dealing with pure and non-pure Python modules, + which are usually not run by themselves but imported by scripts. + + Scripts are files containing Python source code, indended to be started + from the command line. + Distutils doesn't provide much functionality for the scripts: the only + support Distutils gives is to adjust the first line of the script + if it starts with \code{#!} and contains the word ``python'' to refer + to the current interpreter location. + + The \option{scripts} option simply is a list of files to be handled + in this way. + + + \subsection{Listing additional files} + The \option{data\_files} option can be used to specify additional + files needed by the module distribution: configuration files, + data files, anything which does not fit in the previous categories. + + \option{data\_files} specify a sequence of \code{(directory, files)} + pairs in the following way: + \begin{verbatim} + setup(... + data_files=[('bitmaps', ['bm/b1.gif', 'bm/b2.gif']), + ('config', ['cfg/data.cfg'])]) + \end{verbatim} + + Note that you can specify the directory names where the data files + will be installed, but you cannot rename the data files themselves. + + You can specify the \option{data\_files} options as a simple sequence + of files without specifying a target directory, but this is not recommended, + and the \command{install} command will print a warning in this case. + To install data files directly in the target directory, an empty + string should be given as the directory. + + \section{Writing the Setup Configuration File} \label{setup-config} *************** *** 948,955 **** \lineiii{zip}{zip file (\file{.zip})}{(4)} \lineiii{rpm}{RPM}{(5)} \lineiii{srpm}{source RPM}{(5) \XXX{to do!}} ! \lineiii{wininst}{self-extracting ZIP file for Windows}{(2),(6)} ! %\lineiii{wise}{Wise installer for Windows}{(3)} \end{tableiii} \noindent Notes: --- 999,1005 ---- \lineiii{zip}{zip file (\file{.zip})}{(4)} \lineiii{rpm}{RPM}{(5)} \lineiii{srpm}{source RPM}{(5) \XXX{to do!}} ! \lineiii{wininst}{self-extracting ZIP file for Windows}{(2),(4)} \end{tableiii} \noindent Notes: *************** *** 962,969 **** \module{zipfile} module (not part of the standard Python library) \item[(5)] requires external \program{rpm} utility, version 3.0.4 or better (use \code{rpm --version} to find out which version you have) - \item[(6)] \XXX{requirements for \command{bdist\_wininst}?} - %\item[(3)] not implemented yet \end{description} You don't have to use the \command{bdist} command with the --- 1012,1017 ---- *************** *** 980,986 **** \lineii{bdist\_dumb}{tar, ztar, gztar, zip} \lineii{bdist\_rpm}{rpm, srpm} \lineii{bdist\_wininst}{wininst} - %\lineii{bdist\_wise}{wise} \end{tableii} The following sections give details on the individual \command{bdist\_*} --- 1028,1033 ---- *************** *** 1093,1099 **** \XXX{this isn't implemented yet---is it needed?!} You can also specify a custom \file{.spec} file with the ! \longprogramopt{spec-file} option; used in conjunctin with \longprogramopt{spec-only}, this gives you an opportunity to customize the \file{.spec} file manually: \begin{verbatim} --- 1140,1146 ---- \XXX{this isn't implemented yet---is it needed?!} You can also specify a custom \file{.spec} file with the ! \longprogramopt{spec-file} option; used in conjunction with \longprogramopt{spec-only}, this gives you an opportunity to customize the \file{.spec} file manually: \begin{verbatim} *************** *** 1110,1116 **** --- 1157,1194 ---- \subsection{Creating Windows installers} \label{creating-wininst} + Executable Windows installers are the natural format for binary + distributions on Windows. They display a nice GUI interface, display + some information of the module distribution to be installed, taken + from the meta-dada in the setup script, let the user select a few + (currently maybe too few) options, and start or cancel the installation. + Since the meta-data is taken from the setup script, creating + Windows installers is usually as easy as running: + \begin{verbatim} + python setup.py bdist_wininst + \end{verbatim} + or the \command{bdist} command with the \longprogramopt{format} option: + \begin{verbatim} + python setup.py bdist --formats=wininst + \end{verbatim} + + If you have a pure module distribution (only containing pure + Python modules and packages), the resulting installer will be + version independent and have a name like \file{Foo-1.0.win32.exe}. + These installers can even be created on \UNIX{} or MacOS platforms. + + If you have a non-pure distribution, the extensions can only be + created on a Windows platform, and will be Python version dependend. + The installer filename will reflect this and now has the form + \file{Foo-1.0.win32-py2.0.exe}. You have to create a separate installer + for every Python version you want to support. + + The installer will try to compile pure modules into bytecode after + installation on the target system in normal and optimizing mode. + If you don't want this to happen for some reason, you can run + the bdist_wininst command with the \longprogramopt{no-target-compile} and/or + the \longprogramopt{no-target-optimize} option. \section{Examples} \label{examples} *************** *** 1146,1151 **** --- 1224,1231 ---- \subsection{Writing new commands} \label{new-commands} + + \XXX{Would an uninstall command be a good example here?} From thomas.heller@ion-tof.com Mon Feb 19 08:16:01 2001 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Mon Feb 19 08:16:01 2001 Subject: [Distutils] Documentation Message-ID: <004401c09a75$fe3cea30$e000a8c0@thomasnotebook> Here is what I've hacked together over some spare time at the weekend. I would appreciate feedback, as I've told before I'm neither a great author, nor native english speaker, nor anything else. Mostly I've completed (and written) docs for bdist_wininst (where I'm an expertert for), also I've started some stuff for the data_files and scripts option. Should I continue in this way? Would others like to jump in? Thomas Index: dist.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/dist/dist.tex,v retrieving revision 1.29 diff -c -r1.29 dist.tex *** dist.tex 2000/11/22 16:06:16 1.29 --- dist.tex 2001/02/19 13:07:06 *************** *** 53,59 **** Using the Distutils is quite simple, both for module developers and for users/administrators installing third-party modules. As a developer, ! your responsibilites (apart from writing solid, well-documented and well-tested code, of course!) are: \begin{itemize} \item write a setup script (\file{setup.py} by convention) --- 53,59 ---- Using the Distutils is quite simple, both for module developers and for users/administrators installing third-party modules. As a developer, ! your responsibilities (apart from writing solid, well-documented and well-tested code, of course!) are: \begin{itemize} \item write a setup script (\file{setup.py} by convention) *************** *** 148,166 **** will create an executable installer, \file{Foo-1.0.win32.exe}, in the current directory. ! \XXX{not implemented yet} ! (Another way to create executable installers for Windows is with the ! \command{bdist\_wise} command, which uses Wise---the commercial ! installer-generator used to create Python's own installer---to create ! the installer. Wise-based installers are more appropriate for large, ! industrial-strength applications that need the full capabilities of a ! ``real'' installer. \command{bdist\_wininst} creates a self-extracting ! zip file with a minimal user interface, which is enough for small- to ! medium-sized module collections. You'll need to have version XXX of ! Wise installed on your system for the \command{bdist\_wise} command to ! work; it's available from \url{http://foo/bar/baz}.) ! ! Currently (Distutils 0.9.2), the are only other useful built distribution format is RPM, implemented by the \command{bdist\_rpm} command. For example, the following command will create an RPM file called \file{Foo-1.0.noarch.rpm}: --- 148,154 ---- will create an executable installer, \file{Foo-1.0.win32.exe}, in the current directory. ! Currently (Distutils 0.9.2), the only other useful built distribution format is RPM, implemented by the \command{bdist\_rpm} command. For example, the following command will create an RPM file called \file{Foo-1.0.noarch.rpm}: *************** *** 192,198 **** single \file{.py} file (and possibly associated \file{.pyc} and/or \file{.pyo} files). Sometimes referred to as a ``pure module.'' \item[extension module] a module written in the low-level language of ! the Python implemention: C/C++ for Python, Java for JPython. Typically contained in a single dynamically loadable pre-compiled file, e.g. a shared object (\file{.so}) file for Python extensions on \UNIX, a DLL (given the \file{.pyd} extension) for Python extensions --- 180,186 ---- single \file{.py} file (and possibly associated \file{.pyc} and/or \file{.pyo} files). Sometimes referred to as a ``pure module.'' \item[extension module] a module written in the low-level language of ! the Python implementation: C/C++ for Python, Java for JPython. Typically contained in a single dynamically loadable pre-compiled file, e.g. a shared object (\file{.so}) file for Python extensions on \UNIX, a DLL (given the \file{.pyd} extension) for Python extensions *************** *** 240,246 **** \section{Writing the Setup Script} \label{setup-script} ! The setup script is the centre of all activity in building, distributing, and installing modules using the Distutils. The main purpose of the setup script is to describe your module distribution to the Distutils, so that the various commands that operate on your modules --- 228,234 ---- \section{Writing the Setup Script} \label{setup-script} ! The setup script is the center of all activity in building, distributing, and installing modules using the Distutils. The main purpose of the setup script is to describe your module distribution to the Distutils, so that the various commands that operate on your modules *************** *** 290,295 **** --- 278,291 ---- mind that the \emph{absence} of a leading slash indicates a relative path, the opposite of the MacOS convention with colons). + This, of course, only applies to pathnames given to Distutils functions. + If you, for example, use standard python functions such as glob.glob + or os.listdir to specify files, you should be careful to write portable + code instead of hardcoding path separators: + \begin{verbatim} + glob.glob(os.path.join('mydir', 'subdir', '*.html')) + os.listdir(os.path.join('mydir', 'subdir')) + \end{verbatim} \subsection{Listing whole packages} \label{listing-packages} *************** *** 447,454 **** On some platforms, you can include non-source files that are processed by the compiler and included in your extension. Currently, this just ! means Windows resource files for Visual C++. \XXX{get more detail on ! this feature from Thomas Heller!} \subsubsection{Preprocessor options} --- 443,451 ---- On some platforms, you can include non-source files that are processed by the compiler and included in your extension. Currently, this just ! means Windows message text (\file{.mc}) files and resource definition ! (\file{.rc}) files for Visual C++. These will be compiled to binary resource ! (\file{.res}) files and linked into the executable. \subsubsection{Preprocessor options} *************** *** 549,560 **** \end{verbatim} (Again, this sort of non-portable construct should be avoided if you intend to distribute your code.) ! \XXX{still undocumented: extra\_objects, extra\_compile\_args, ! extra\_link\_args, export\_symbols---none of which are frequently ! needed, some of which might be completely unnecessary!} \section{Writing the Setup Configuration File} \label{setup-config} --- 546,611 ---- \end{verbatim} (Again, this sort of non-portable construct should be avoided if you intend to distribute your code.) + + \XXX{THe Should mention clib libraries here or somewhere else!} + + \subsubsection{Other options} + + There are still some other options which can be used to handle special + cases. + + The \option{extra\_objects} option is a list of object files to be passed + to the linker. These files must not have extensions, the default extension + for the compiler is used. ! \option{extra\_compile\_args} and \option{extra\_link\_args} can be used ! to specify additional command line options for the compiler resp. ! the linker command line. + \option{export\_symbols} is only useful on windows, it can contain a list + of symbols (funtions or variables) to be exported. This option + is not needed when building compiled extensions: the \code{initmodule} + function will automatically be added to the exported symbols list + by Distutils. + \subsection{Listing scripts} + So far we have been dealing with pure and non-pure Python modules, + which are usually not run by themselves but imported by scripts. + + Scripts are files containing Python source code, indended to be started + from the command line. + Distutils doesn't provide much functionality for the scripts: the only + support Distutils gives is to adjust the first line of the script + if it starts with \code{#!} and contains the word ``python'' to refer + to the current interpreter location. + + The \option{scripts} option simply is a list of files to be handled + in this way. + + + \subsection{Listing additional files} + The \option{data\_files} option can be used to specify additional + files needed by the module distribution: configuration files, + data files, anything which does not fit in the previous categories. + + \option{data\_files} specify a sequence of \code{(directory, files)} + pairs in the following way: + \begin{verbatim} + setup(... + data_files=[('bitmaps', ['bm/b1.gif', 'bm/b2.gif']), + ('config', ['cfg/data.cfg'])]) + \end{verbatim} + + Note that you can specify the directory names where the data files + will be installed, but you cannot rename the data files themselves. + + You can specify the \option{data\_files} options as a simple sequence + of files without specifying a target directory, but this is not recommended, + and the \command{install} command will print a warning in this case. + To install data files directly in the target directory, an empty + string should be given as the directory. + + \section{Writing the Setup Configuration File} \label{setup-config} *************** *** 948,955 **** \lineiii{zip}{zip file (\file{.zip})}{(4)} \lineiii{rpm}{RPM}{(5)} \lineiii{srpm}{source RPM}{(5) \XXX{to do!}} ! \lineiii{wininst}{self-extracting ZIP file for Windows}{(2),(6)} ! %\lineiii{wise}{Wise installer for Windows}{(3)} \end{tableiii} \noindent Notes: --- 999,1005 ---- \lineiii{zip}{zip file (\file{.zip})}{(4)} \lineiii{rpm}{RPM}{(5)} \lineiii{srpm}{source RPM}{(5) \XXX{to do!}} ! \lineiii{wininst}{self-extracting ZIP file for Windows}{(2),(4)} \end{tableiii} \noindent Notes: *************** *** 962,969 **** \module{zipfile} module (not part of the standard Python library) \item[(5)] requires external \program{rpm} utility, version 3.0.4 or better (use \code{rpm --version} to find out which version you have) - \item[(6)] \XXX{requirements for \command{bdist\_wininst}?} - %\item[(3)] not implemented yet \end{description} You don't have to use the \command{bdist} command with the --- 1012,1017 ---- *************** *** 980,986 **** \lineii{bdist\_dumb}{tar, ztar, gztar, zip} \lineii{bdist\_rpm}{rpm, srpm} \lineii{bdist\_wininst}{wininst} - %\lineii{bdist\_wise}{wise} \end{tableii} The following sections give details on the individual \command{bdist\_*} --- 1028,1033 ---- *************** *** 1093,1099 **** \XXX{this isn't implemented yet---is it needed?!} You can also specify a custom \file{.spec} file with the ! \longprogramopt{spec-file} option; used in conjunctin with \longprogramopt{spec-only}, this gives you an opportunity to customize the \file{.spec} file manually: \begin{verbatim} --- 1140,1146 ---- \XXX{this isn't implemented yet---is it needed?!} You can also specify a custom \file{.spec} file with the ! \longprogramopt{spec-file} option; used in conjunction with \longprogramopt{spec-only}, this gives you an opportunity to customize the \file{.spec} file manually: \begin{verbatim} *************** *** 1110,1116 **** --- 1157,1194 ---- \subsection{Creating Windows installers} \label{creating-wininst} + Executable Windows installers are the natural format for binary + distributions on Windows. They display a nice GUI interface, display + some information of the module distribution to be installed, taken + from the meta-dada in the setup script, let the user select a few + (currently maybe too few) options, and start or cancel the installation. + Since the meta-data is taken from the setup script, creating + Windows installers is usually as easy as running: + \begin{verbatim} + python setup.py bdist_wininst + \end{verbatim} + or the \command{bdist} command with the \longprogramopt{format} option: + \begin{verbatim} + python setup.py bdist --formats=wininst + \end{verbatim} + + If you have a pure module distribution (only containing pure + Python modules and packages), the resulting installer will be + version independent and have a name like \file{Foo-1.0.win32.exe}. + These installers can even be created on \UNIX{} or MacOS platforms. + + If you have a non-pure distribution, the extensions can only be + created on a Windows platform, and will be Python version dependend. + The installer filename will reflect this and now has the form + \file{Foo-1.0.win32-py2.0.exe}. You have to create a separate installer + for every Python version you want to support. + + The installer will try to compile pure modules into bytecode after + installation on the target system in normal and optimizing mode. + If you don't want this to happen for some reason, you can run + the bdist_wininst command with the \longprogramopt{no-target-compile} and/or + the \longprogramopt{no-target-optimize} option. \section{Examples} \label{examples} *************** *** 1146,1151 **** --- 1224,1231 ---- \subsection{Writing new commands} \label{new-commands} + + \XXX{Would an uninstall command be a good example here?} From akuchlin@mems-exchange.org Mon Feb 19 11:06:02 2001 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Mon Feb 19 11:06:02 2001 Subject: [Distutils] Documentation In-Reply-To: <004401c09a75$fe3cea30$e000a8c0@thomasnotebook>; from thomas.heller@ion-tof.com on Mon, Feb 19, 2001 at 02:14:58PM +0100 References: <004401c09a75$fe3cea30$e000a8c0@thomasnotebook> Message-ID: <20010219110458.A27070@ute.cnri.reston.va.us> On Mon, Feb 19, 2001 at 02:14:58PM +0100, Thomas Heller wrote: >Should I continue in this way? Would others like to jump in? Good work; I've noted a few minor corrections, but go ahead and check in your changes. I'll probably make additional changes after you check them in. --amk >! The setup script is the centre of all activity in building, >! The setup script is the center of all activity in building, 'centre' is the British spelling, not a typo; Canadians can use either spelling, but Greg seems to prefer the British variant. >+ If you, for example, use standard python functions such as glob.glob >+ or os.listdir to specify files, you should be careful to write portable \function{glob.glob} ... \function{os.listdir} >+ \XXX{THe Should mention clib libraries here or somewhere else!} ^^^ delete? >+ to the linker. These files must not have extensions, the default extension ^ insert 'as', 'because' >+ for the compiler is used. >+ \option{export\_symbols} is only useful on windows, it can contain a list Windows, containing a list >+ of symbols (funtions or variables) to be exported. This option ^c From mal@lemburg.com Mon Feb 19 11:20:02 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Mon Feb 19 11:20:02 2001 Subject: [Distutils] Adding --compiler pass-through option to install References: <3A8E879F.B555BEE6@lemburg.com> <20010218175729.A26452@ute.cnri.reston.va.us> <3A910A37.B6FC6F2D@gmx.de> Message-ID: <3A91476D.45ABE705@lemburg.com> Rene Liebscher wrote: > > Andrew Kuchling wrote: > > > > On Sat, Feb 17, 2001 at 03:15:59PM +0100, M.-A. Lemburg wrote: > > >Wouldn't it make sense to add a --compiler pass-through option > > >to the install command ? > > > > IMHO that leads down the slippery slope of having to pass everything > > through; --libraries and --debug and --define &c. would also all have > > to be passed through, and applied to the proper command. If you need > > special arguments, run the passes separately. > > > If you have to specify it for every package you compile, because > you want to use a particular compiler, then you should write > a personal configuration file as described in > http://www.python.org/sigs/distutils-sig/doc/inst/sec:config-filenames.html Good point. So adding a section [build] compiler = cygwin in Lib\distutils\pydistutils.cfg should take care of such a setting once and for all. Still, someone reported that splitting the build and install steps in two runs of distutils doesn't give the expected behaviour, so we may have a more difficult problem to solve here... Would it work to put multiple commands on a single distutils command line call (associating the options with the command immediatly preceding it) ?! python setup.py build --compiler=foobar install -- Marc-Andre Lemburg ______________________________________________________________________ Company: http://www.egenix.com/ Consulting: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From R.Liebscher@gmx.de Mon Feb 19 11:39:01 2001 From: R.Liebscher@gmx.de (Rene Liebscher) Date: Mon Feb 19 11:39:01 2001 Subject: [Distutils] Adding --compiler pass-through option to install References: <3A8E879F.B555BEE6@lemburg.com> <20010218175729.A26452@ute.cnri.reston.va.us> <3A910A37.B6FC6F2D@gmx.de> <3A91476D.45ABE705@lemburg.com> Message-ID: <3A914BCA.D124A68F@gmx.de> "M.-A. Lemburg" wrote: > > Rene Liebscher wrote: > > > > Andrew Kuchling wrote: > > > > > > On Sat, Feb 17, 2001 at 03:15:59PM +0100, M.-A. Lemburg wrote: > > > >Wouldn't it make sense to add a --compiler pass-through option > > > >to the install command ? > > > > > > IMHO that leads down the slippery slope of having to pass everything > > > through; --libraries and --debug and --define &c. would also all have > > > to be passed through, and applied to the proper command. If you need > > > special arguments, run the passes separately. > > > > > If you have to specify it for every package you compile, because > > you want to use a particular compiler, then you should write > > a personal configuration file as described in > > http://www.python.org/sigs/distutils-sig/doc/inst/sec:config-filenames.html > > Good point. So adding a section > > [build] > compiler = cygwin > > in Lib\distutils\pydistutils.cfg should take care of such a > setting once and for all. > > Still, someone reported that splitting the build and install > steps in two runs of distutils doesn't give the expected behaviour, > so we may have a more difficult problem to solve here... > > Would it work to put multiple commands on a single distutils > command line call (associating the options with the command > immediatly preceding it) ?! > > python setup.py build --compiler=foobar install > If you want to do an install where the intern build uses a particular compiler then python setup.py install build --compiler=foobar should work. Each option is for the last mentioned command, but only the first command is really executed. If it executes internal other commands then they have already set their options. Kind regards Rene Liebscher From mal@lemburg.com Mon Feb 19 12:40:03 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Mon Feb 19 12:40:03 2001 Subject: [Distutils] Adding --compiler pass-through option to install References: <3A8E879F.B555BEE6@lemburg.com> <20010218175729.A26452@ute.cnri.reston.va.us> <3A910A37.B6FC6F2D@gmx.de> <3A91476D.45ABE705@lemburg.com> <3A914BCA.D124A68F@gmx.de> Message-ID: <3A915A2D.5B9F2100@lemburg.com> Rene Liebscher wrote: > > "M.-A. Lemburg" wrote: > > > > Rene Liebscher wrote: > > > > > > Andrew Kuchling wrote: > > > > > > > > On Sat, Feb 17, 2001 at 03:15:59PM +0100, M.-A. Lemburg wrote: > > > > >Wouldn't it make sense to add a --compiler pass-through option > > > > >to the install command ? > > > > > > > > IMHO that leads down the slippery slope of having to pass everything > > > > through; --libraries and --debug and --define &c. would also all have > > > > to be passed through, and applied to the proper command. If you need > > > > special arguments, run the passes separately. > > > > > > > If you have to specify it for every package you compile, because > > > you want to use a particular compiler, then you should write > > > a personal configuration file as described in > > > http://www.python.org/sigs/distutils-sig/doc/inst/sec:config-filenames.html > > > > Good point. So adding a section > > > > [build] > > compiler = cygwin > > > > in Lib\distutils\pydistutils.cfg should take care of such a > > setting once and for all. > > > > Still, someone reported that splitting the build and install > > steps in two runs of distutils doesn't give the expected behaviour, > > so we may have a more difficult problem to solve here... > > > > Would it work to put multiple commands on a single distutils > > command line call (associating the options with the command > > immediatly preceding it) ?! > > > > python setup.py build --compiler=foobar install > > > If you want to do an install where the intern build uses a particular > compiler then > > python setup.py install build --compiler=foobar > > should work. > > Each option is for the last mentioned command, but only the first > command is really executed. If it executes internal other commands > then they have already set their options. Thanks for the insight. Looks like Greg has borrowed the time machine from Guido here ;-) Still, I would find the scheme: scan the options and commands from left to right and execute then in the same order more intuitive -- but maybe just me... it seems too late to change anyway. -- Marc-Andre Lemburg ______________________________________________________________________ Company: http://www.egenix.com/ Consulting: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From thomas.heller@ion-tof.com Mon Feb 19 12:52:01 2001 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Mon Feb 19 12:52:01 2001 Subject: [Distutils] Documentation References: <004401c09a75$fe3cea30$e000a8c0@thomasnotebook> <20010219110458.A27070@ute.cnri.reston.va.us> Message-ID: <026f01c09a9c$a5c969b0$e000a8c0@thomasnotebook> Andrew: > On Mon, Feb 19, 2001 at 02:14:58PM +0100, Thomas Heller wrote: > >Should I continue in this way? Would others like to jump in? > > Good work; I've noted a few minor corrections, but go ahead and check > in your changes. I'll probably make additional changes after you > check them in. > > --amk > > >! The setup script is the centre of all activity in building, > >! The setup script is the center of all activity in building, > > 'centre' is the British spelling, not a typo; Canadians can use either > spelling, but Greg seems to prefer the British variant. Well, I wasn_t sure about that, but ispell suggested 'center', so at least my feeling was right... Thanks, Thomas From thomas.heller@ion-tof.com Mon Feb 19 13:03:13 2001 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Mon Feb 19 13:03:13 2001 Subject: [Distutils] Adding --compiler pass-through option to install References: <3A8E879F.B555BEE6@lemburg.com> <20010218175729.A26452@ute.cnri.reston.va.us> <3A910A37.B6FC6F2D@gmx.de> <3A91476D.45ABE705@lemburg.com> <3A914BCA.D124A68F@gmx.de> Message-ID: <02c701c09a9e$26d38300$e000a8c0@thomasnotebook> From: "Rene Liebscher" : > If you want to do an install where the intern build uses a particular > compiler then > > python setup.py install build --compiler=foobar > > should work. > > Each option is for the last mentioned command, but only the first > command is really executed. If it executes internal other commands > then they have already set their options. > But why, if this would be true, does python setup.py build --debug install install anything? Thomas From nhv@cape.com Mon Feb 19 13:08:01 2001 From: nhv@cape.com (Norman Vine) Date: Mon Feb 19 13:08:01 2001 Subject: [Distutils] Adding --compiler pass-through option to install In-Reply-To: <3A914BCA.D124A68F@gmx.de> Message-ID: <001301c09a9e$9c11dae0$a300a8c0@nhv> Rene Liebscher writes: > >"M.-A. Lemburg" wrote: >> >> Rene Liebscher wrote: >> > >> > > >> > If you have to specify it for every package you compile, because >> > you want to use a particular compiler, then you should write >> > a personal configuration file as described in >> > http://www.python.org/sigs/distutils-sig/doc/inst/sec:config-filenames.html > >> Good point. So adding a section > >> [build] >> compiler = cygwin >> >> in Lib\distutils\pydistutils.cfg should take care of such a >> setting once and for all. This seems to work with cygwin at least < requires this recent patch though :-) > http://sourceforge.net/patch/?func=detailpatch&patch_id=103868&group_id=5470 >If you want to do an install where the intern build uses a particular >compiler then >python setup.py install build --compiler=foobar This works without the aforementioed patch :-) Cheers Norman Vine From kleistereimer@gmx.de Mon Feb 19 13:17:01 2001 From: kleistereimer@gmx.de (kleistereimer@gmx.de) Date: Mon Feb 19 13:17:01 2001 Subject: [Distutils] docs/examples for building windows modules (dll's) from c++. distutils ??? Message-ID: <11418.982606605@www10.gmx.net> hi! i'm a c++ professional, but i have converted to python now. (wx..) my 2 problems are: 1. i have lot's of hardware-dependend c++ classes, i need to extend python with. 2. i need to make a (binary) installer for linux and windows. (at best not releasing the sources, and including a python interpreter already. (i can do this with a modified py2exe already)) I suppose the distutils will (at least) help with both (?) problems. (i like my software to be standart-conform) 1: there are examples and docs to extend python with c-code, but not for c++ classes. so how to wrap c++ code to make it callable from python? (c++ code means a complete class hierarchy in aprox 30 files which contains one interface class and one iterator class on top, which both needs to be accessible from python) 2: next thing is how to compile the wrapped c++. i found references to a 'compile.py' which generates vc6 project-files. but it's not available anymore. (use distutils everyone tells me..) and i wont use outdated tools.. the documentation of distutils is not very clear. and the examples-pages are empty. all i need is a minimal example where a c++ class is interfaced to python, and compiled to a dll using bc, or vc. at best all driven by distutils to make it standart-conform. distutils seem to encapsulate compilers, so maybe i'm able to generate the dll using distutils? or am i completely wrong? please point me to some information! THANK YOU!! -- Sent through GMX FreeMail - http://www.gmx.net From thomas.heller@ion-tof.com Mon Feb 19 13:26:48 2001 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Mon Feb 19 13:26:48 2001 Subject: [Distutils] docs/examples for building windows modules (dll's) from c++. distutils ??? References: <11418.982606605@www10.gmx.net> Message-ID: <031301c09aa1$51a95d40$e000a8c0@thomasnotebook> kleistereimer wrote: > hi! > > i'm a c++ professional, but i have converted to python now. (wx..) > my 2 problems are: > > 1. i have lot's of hardware-dependend c++ classes, i need to extend > python with. > 2. i need to make a (binary) installer for linux and windows. > (at best not releasing the sources, and including a python interpreter > already. (i can do this with a modified py2exe already)) As the author of py2exe I would certainly like to hear more about this! > > I suppose the distutils will (at least) help with both (?) problems. > (i like my software to be standart-conform) > > 1: > there are examples and docs to extend python with c-code, > but not for c++ classes. > so how to wrap c++ code to make it callable from python? > (c++ code means a complete class hierarchy in aprox 30 files > which contains one interface class and one iterator class on top, > which both needs to be accessible from python) > > 2: > next thing is how to compile the wrapped c++. > Take a look at SWIG: http://www.swig.org/ Distutils contains (in an early stage) support for swig. Thomas From akuchlin@mems-exchange.org Mon Feb 19 14:10:00 2001 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Mon Feb 19 14:10:00 2001 Subject: [Distutils] Adding --compiler pass-through option to install In-Reply-To: <02c701c09a9e$26d38300$e000a8c0@thomasnotebook>; from thomas.heller@ion-tof.com on Mon, Feb 19, 2001 at 07:02:41PM +0100 References: <3A8E879F.B555BEE6@lemburg.com> <20010218175729.A26452@ute.cnri.reston.va.us> <3A910A37.B6FC6F2D@gmx.de> <3A91476D.45ABE705@lemburg.com> <3A914BCA.D124A68F@gmx.de> <02c701c09a9e$26d38300$e000a8c0@thomasnotebook> Message-ID: <20010219140905.A1287@ute.cnri.reston.va.us> On Mon, Feb 19, 2001 at 07:02:41PM +0100, Thomas Heller wrote: >From: "Rene Liebscher" : >> Each option is for the last mentioned command, but only the first >> command is really executed. If it executes internal other commands >> then they have already set their options. >> >But why, if this would be true, does >python setup.py build --debug install >install anything? Rene's explanation isn't quite correct; commands are run from left to right, but there's a cache to prevent commands from being run twice, so 'build' is run, and then 'install', which doesn't re-run 'build'. See the run_commands()/run_command() methods in the Distribution class. --amk From Jason.Tishler@dothill.com Mon Feb 19 14:38:02 2001 From: Jason.Tishler@dothill.com (Jason Tishler) Date: Mon Feb 19 14:38:02 2001 Subject: [Distutils] Re: [4suite] Re: Compiling 4Suite 0.10.2beta on Windows In-Reply-To: <200102180955.f1I9t3700868@mira.informatik.hu-berlin.de>; from martin@loewis.home.cs.tu-berlin.de on Sun, Feb 18, 2001 at 10:55:03AM +0100 References: <200102180019.f1I0JjQ10902@mira.informatik.hu-berlin.de> <20010217224412.A226@dothill.com> <200102180955.f1I9t3700868@mira.informatik.hu-berlin.de> Message-ID: <20010219143652.O211@dothill.com> Martin, On Sun, Feb 18, 2001 at 10:55:03AM +0100, Martin v. Loewis wrote: > > I wouldn't say that using Cygwin is a bad idea -- I'd rather state that > > if you do so then your extension is dependent on Cygwin's POSIX emulation > > layer (i.e., cygwin1.dll) and that you need Cygwin Python to execute it. > > That's what I meant - if you have an MSVC-built Python, than a > cygwin-built extension probably won't work, will it? In my limited experimentation, Cygwin extensions do not work with a MSVC Python. Given that they link with the Cygwin Python import library (and import from the Cygwin Python DLL) as opposed to a MSVC one, I didn't really expect it to work. However, Rene seems to indicate that at least "simple" Cygwin extensions should work with MSVC Python in the following: On Mon, Feb 19, 2001 at 01:02:03PM +0100, Rene Liebscher wrote: > With --compiler=mingw32 compiled extensions run without problems on an > MSVC built Python. > > Some simpler examples also worked when compiled with --compiler=cygwin > (I never tested it thorougly because I prefer not to need an additional > dll.) > > However, Python 2.1a2 builds OOTB under Cygwin and supports both the > > traditional (i.e. Misc/Makefile.pre.in) and distutils methods of > > building shared extensions. > > That's good to hear. So what about using Cygwin to build > distutils-made extensions when the Python installation was built with > MSVC? The most recent Cygwin gcc release (i.e., gcc-2.95.2-7) nows binds executables against msvcrt.dll instead of crtdll.dll when the Mingw mode (i.e., "-mno-cygwin" option) is specified. IMO, using Cygwin's gcc in Mingw mode to built MSVC Python extensions is now a viable alternative to the real Mingw gcc. I'm not that knowledgeable with distutils, so again I will quote from Rene's email: On Mon, Feb 19, 2001 at 01:02:03PM +0100, Rene Liebscher wrote: > For MinGW and Cygwin in MinGW-mode it is --compiler=mingw32 >From the above, it sounds like distutils is already setup to use Cygwin gcc in Mingw mode, so it seems like we are good to go. Jason -- Jason Tishler Director, Software Engineering Phone: +1 (732) 264-8770 x235 Dot Hill Systems Corp. Fax: +1 (732) 264-8798 82 Bethany Road, Suite 7 Email: Jason.Tishler@dothill.com Hazlet, NJ 07730 USA WWW: http://www.dothill.com From akuchlin@mems-exchange.org Tue Feb 20 12:13:24 2001 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Tue Feb 20 12:13:24 2001 Subject: [Distutils] What of Siphon? Message-ID: Suchandra Thapa's Siphon code is now available. http://sourceforge.net/project/showfiles.php?group_id=20509&release_id=24001 Comments? Let's not have the issue go to sleep again! --amk From Jason.Tishler@dothill.com Tue Feb 20 12:36:01 2001 From: Jason.Tishler@dothill.com (Jason Tishler) Date: Tue Feb 20 12:36:01 2001 Subject: [Distutils] Re: [4suite] Re: Compiling 4Suite 0.10.2beta on Windows In-Reply-To: <20010219143652.O211@dothill.com>; from Jason.Tishler@dothill.com on Mon, Feb 19, 2001 at 02:36:52PM -0500 References: <200102180019.f1I0JjQ10902@mira.informatik.hu-berlin.de> <20010217224412.A226@dothill.com> <200102180955.f1I9t3700868@mira.informatik.hu-berlin.de> <20010219143652.O211@dothill.com> Message-ID: <20010220123432.F512@dothill.com> Martin, On Mon, Feb 19, 2001 at 02:36:52PM -0500, Jason Tishler wrote: > On Sun, Feb 18, 2001 at 10:55:03AM +0100, Martin v. Loewis wrote: > > That's good to hear. So what about using Cygwin to build > > distutils-made extensions when the Python installation was built with > > MSVC? > > The most recent Cygwin gcc release (i.e., gcc-2.95.2-7) nows binds > executables against msvcrt.dll instead of crtdll.dll when the Mingw mode > (i.e., "-mno-cygwin" option) is specified. IMO, using Cygwin's gcc in > Mingw mode to built MSVC Python extensions is now a viable alternative > to the real Mingw gcc. I spoke (one day) too soon. When I was originally writing the above, I started with phrases like "The next Cygwin gcc release" and "The as yet unannounced Cygwin gcc release," but I decided to take a chance. Unfortunately, the Red Hat folks just yanked the Cygwin gcc 2.95.2-7 release yesterday due to other problems. The current Cygwin gcc release (i.e., 2.95.2-6) still binds against crtdll.dll. Sorry, if my post caused any heartaches. Jason -- Jason Tishler Director, Software Engineering Phone: +1 (732) 264-8770 x235 Dot Hill Systems Corp. Fax: +1 (732) 264-8798 82 Bethany Road, Suite 7 Email: Jason.Tishler@dothill.com Hazlet, NJ 07730 USA WWW: http://www.dothill.com From mal@lemburg.com Tue Feb 20 13:27:29 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Tue Feb 20 13:27:29 2001 Subject: [Distutils] What of Siphon? References: Message-ID: <3A92B6E1.3AD4789C@lemburg.com> Andrew Kuchling wrote: > > Suchandra Thapa's Siphon code is now available. > > http://sourceforge.net/project/showfiles.php?group_id=20509&release_id=24001 > > Comments? Let's not have the issue go to sleep again! Hmm, no docs, no summary, no home page... what is this Siphon thingie ? ;-) -- Marc-Andre Lemburg ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Pages: http://www.lemburg.com/python/ From phrxy@csv.warwick.ac.uk Tue Feb 20 14:19:00 2001 From: phrxy@csv.warwick.ac.uk (John J. Lee) Date: Tue Feb 20 14:19:00 2001 Subject: [Distutils] Re: [4suite] Re: Compiling 4Suite 0.10.2beta on Windows In-Reply-To: <20010220123432.F512@dothill.com> Message-ID: On Tue, 20 Feb 2001, Jason Tishler wrote: [...] > The current Cygwin gcc release (i.e., 2.95.2-6) still binds against > crtdll.dll. Sorry, if my post caused any heartaches. I never managed to sort out all the details (anyone like to recommend a source of info that explains DLLs (and .so's too for that matter) properly?), but just to make completely clear: the -c mingw32 option to setup.py does appear to work fine at the moment. You don't need to get mingw32 separately, just get cygwin (unless you don't want any of the cygwin stuff, of course). John From guido@digicool.com Tue Feb 20 22:19:41 2001 From: guido@digicool.com (Guido van Rossum) Date: Tue Feb 20 22:19:41 2001 Subject: [Distutils] Python Siphon name change please? Message-ID: <200102210300.WAA08543@cj20424-a.reston1.va.home.com> Sorry for the introsion. I just found out that there's a thing called Python-Siphon being discussed on the distutils-sig (to which I'm not subscribed), which is CPAN functionality. I believe that falls under the Catalog SIG's charter too (to which I *am* subscribed, I believe -- but it's been dead since last November), so I'mm ccing there. I love the concept. But please don't use that name. I searched Google for "python siphon" and found out it is a technical term for something used to siphon water into or out of aquariums (?). That's cute, but makes it hard to find through web searches, and that's not what we want for a CPAN thingie! :-) Feel free to flame me if this is inappropriate! --Guido van Rossum (home page: http://www.python.org/~guido/) From robin@jessikat.fsnet.co.uk Wed Feb 21 10:02:42 2001 From: robin@jessikat.fsnet.co.uk (Robin Becker) Date: Wed Feb 21 10:02:42 2001 Subject: [Distutils] Python Siphon name change please? In-Reply-To: <200102210300.WAA08543@cj20424-a.reston1.va.home.com> References: <200102210300.WAA08543@cj20424-a.reston1.va.home.com> Message-ID: In article <200102210300.WAA08543@cj20424-a.reston1.va.home.com>, Guido van Rossum writes >Sorry for the introsion. I just found out that there's a thing called >Python-Siphon being discussed on the distutils-sig (to which I'm not >subscribed), which is CPAN functionality. I believe that falls under >the Catalog SIG's charter too (to which I *am* subscribed, I believe >-- but it's been dead since last November), so I'mm ccing there. > >I love the concept. But please don't use that name. I searched >Google for "python siphon" and found out it is a technical term for >something used to siphon water into or out of aquariums (?). That's >cute, but makes it hard to find through web searches, and that's not >what we want for a CPAN thingie! :-) > >Feel free to flame me if this is inappropriate! > >--Guido van Rossum (home page: http://www.python.org/~guido/) Well there was a very long discussion about this on clp, but I guess dictators don't listen to ordinary folk. I guess the common fate of dictators is that the masses eventually catch up with them in surprising ways. Siphon seems quite a good name in terms of the etymology and rhyme. If all new decisions on module name have to be referred to google it'll be a bit limiting. Feel free to post an alternate and then we can submit it to google. -- Robin Becker From alex@ank-sia.com Wed Feb 21 10:32:08 2001 From: alex@ank-sia.com (alexander smishlajev) Date: Wed Feb 21 10:32:08 2001 Subject: [Distutils] Re: [4suite] Re: Compiling 4Suite 0.10.2beta on Windows References: <200102180019.f1I0JjQ10902@mira.informatik.hu-berlin.de> Message-ID: <3A93B452.BABC0A87@ank-sia.com> "Martin v. Loewis" wrote: > > It would probably be good if some volunteer could provide a mingw > import library of python20.dll, and if python 2.1 could provide > such an import library as a battery. i have put the library building instructions to http://www.zope.org/Members/als/tips/win32_mingw_modules ready-to-use import library for mingw32 is in http://www.zope.org/Members/als/libpython20.zip best wishes, alex. From akuchlin@mems-exchange.org Wed Feb 21 11:19:07 2001 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Wed Feb 21 11:19:07 2001 Subject: [Distutils] Python Siphon name change please? In-Reply-To: ; from robin@jessikat.fsnet.co.uk on Wed, Feb 21, 2001 at 09:11:52AM +0000 References: <200102210300.WAA08543@cj20424-a.reston1.va.home.com> Message-ID: <20010221100503.B29605@ute.cnri.reston.va.us> On Wed, Feb 21, 2001 at 09:11:52AM +0000, Robin Becker wrote: >Siphon seems quite a good name in terms of the etymology and rhyme. If >all new decisions on module name have to be referred to google it'll be >a bit limiting. And frankly, quibbling about the name when we don't actually have an implementation is premature! --amk From Jason.Tishler@dothill.com Wed Feb 21 13:11:02 2001 From: Jason.Tishler@dothill.com (Jason Tishler) Date: Wed Feb 21 13:11:02 2001 Subject: [Distutils] Re: [4suite] Re: Compiling 4Suite 0.10.2beta on Windows In-Reply-To: ; from phrxy@csv.warwick.ac.uk on Tue, Feb 20, 2001 at 07:18:42PM +0000 References: <20010220123432.F512@dothill.com> Message-ID: <20010221131021.C738@dothill.com> John, On Tue, Feb 20, 2001 at 07:18:42PM +0000, John J. Lee wrote: > > On Tue, 20 Feb 2001, Jason Tishler wrote: > [...] > > The current Cygwin gcc release (i.e., 2.95.2-6) still binds against > > crtdll.dll. Sorry, if my post caused any heartaches. > > [...] > but just to make completely clear: the -c mingw32 option to > setup.py does appear to work fine at the moment. > > You don't need to get mingw32 separately, just get cygwin (unless you > don't want any of the cygwin stuff, of course). If one uses the current Cygwin gcc in Mingw mode (i.e., -mno-cygwin), then their extensions will use a different C runtime (i.e., crtdll.dll) than the one used by the Win32 Python interpreter (i.e., msvcrt.dll). IIRC, this is bound to cause problems such as crashing the interpreter. Jason -- Jason Tishler Director, Software Engineering Phone: +1 (732) 264-8770 x235 Dot Hill Systems Corp. Fax: +1 (732) 264-8798 82 Bethany Road, Suite 7 Email: Jason.Tishler@dothill.com Hazlet, NJ 07730 USA WWW: http://www.dothill.com From Jason.Tishler@dothill.com Wed Feb 21 14:10:01 2001 From: Jason.Tishler@dothill.com (Jason Tishler) Date: Wed Feb 21 14:10:01 2001 Subject: [Distutils] Cygwin Python Distutils Build Problem Message-ID: <20010221140917.F738@dothill.com> --futHohkuO/V6j/1Y Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Wed, Feb 21, 2001 at 03:42:04PM +0100, M.-A. Lemburg wrote: > I don't want distutils to go into the beta with this known > problem. The patch that you posted seems to be the way to go, > so I am very much for putting it into the core unless is breaks > something else which doesn't seem to be the case. > > Anyway, please post the patch and a summary of the problem to > distutils -- maybe Rene knows more about this. Marc Lemburg recently committed a patch to facilitate the building of his mx Extensions under Cygwin. I have attached his patch as the first attachment for easy access. See the following for details, if interested: http://sourceforge.net/patch/?func=detailpatch&patch_id=103868&group_id=5470 Unfortunately, this patch breaks the Cygwin Python build. Specifically, the standard (shared extension) modules no longer build. See the second attachment for details, if interested. Being very inexperienced with distutils, I humbly offered the following WAG patch which corrected the Cygwin Python build problem. The purpose of this patch is to (re)enable distutils to use the Makefile variables such as CCSHARED, LDSHARED, etc. when building shared extensions under Cygwin just like on all other UNIXes. Please see the third attachment for details. We have the following questions: 1. Is my patch the best way to resolve this issue? 2. The current cygwinccompiler.py (around line 97) has some hardcoded gcc options (i.e., using -static instead -shared) that appear to be out of date. Should these be revisited too? Hopefully, more knowledgeable people can help us resolve these issues before the first beta. Thanks, Jason -- Jason Tishler Director, Software Engineering Phone: +1 (732) 264-8770 x235 Dot Hill Systems Corp. Fax: +1 (732) 264-8798 82 Bethany Road, Suite 7 Email: Jason.Tishler@dothill.com Hazlet, NJ 07730 USA WWW: http://www.dothill.com --futHohkuO/V6j/1Y Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="ccompiler.py.patch" Index: ccompiler.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/distutils/ccompiler.py,v retrieving revision 1.37 retrieving revision 1.38 diff -u -r1.37 -r1.38 --- ccompiler.py 2001/01/16 03:10:43 1.37 +++ ccompiler.py 2001/02/19 09:20:04 1.38 @@ -5,9 +5,9 @@ # created 1999/07/05, Greg Ward -__revision__ = "$Id: ccompiler.py,v 1.37 2001/01/16 03:10:43 akuchling Exp $" +__revision__ = "$Id: ccompiler.py,v 1.38 2001/02/19 09:20:04 lemburg Exp $" -import sys, os +import sys, os, re from types import * from copy import copy from distutils.errors import * @@ -835,13 +835,45 @@ # class CCompiler -# Map a platform ('posix', 'nt') to the default compiler type for -# that platform. -default_compiler = { 'posix': 'unix', - 'nt': 'msvc', - 'mac': 'mwerks', - } +# Map a sys.platform/os.name ('posix', 'nt') to the default compiler +# type for that platform. Keys are interpreted as re match +# patterns. Order is important; platform mappings are preferred over +# OS names. +_default_compilers = ( + # Platform string mappings + ('cygwin.*', 'cygwin'), + + # OS name mappings + ('posix', 'unix'), + ('nt', 'msvc'), + ('mac', 'mwerks'), + + ) + +def get_default_compiler(osname=None, platform=None): + + """ Determine the default compiler to use for the given platform. + + osname should be one of the standard Python OS names (i.e. the + ones returned by os.name) and platform the common value + returned by sys.platform for the platform in question. + + The default values are os.name and sys.platform in case the + parameters are not given. + + """ + if osname is None: + osname = os.name + if platform is None: + platform = sys.platform + for pattern, compiler in _default_compilers: + if re.match(pattern, platform) is not None or \ + re.match(pattern, osname) is not None: + return compiler + # Default to Unix compiler + return 'unix' + # Map compiler types to (module_name, class_name) pairs -- ie. where to # find the code that implements an interface to this compiler. (The module # is assumed to be in the 'distutils' package.) @@ -896,7 +928,7 @@ try: if compiler is None: - compiler = default_compiler[plat] + compiler = get_default_compiler(plat) (module_name, class_name, long_description) = compiler_class[compiler] except KeyError: --futHohkuO/V6j/1Y Content-Type: message/rfc822 Content-Disposition: inline Date: Mon, 19 Feb 2001 17:10:03 -0500 From: Jason Tishler To: "M.-A. Lemburg" Cc: nhv@cape.com Bcc: jt+misc.copy@ny.dothill.com Subject: Re: [Distutils] Beta-release of installers for mx Extensions Message-ID: <20010219171003.R211@dothill.com> References: <000a01c09a18$a57d0360$a300a8c0@nhv> <3A914E98.8FA6FF51@lemburg.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <3A914E98.8FA6FF51@lemburg.com>; from mal@lemburg.com on Mon, Feb 19, 2001 at 05:49:28PM +0100 Organization: Dot Hill Systems Corp. Marc, On Mon, Feb 19, 2001 at 05:49:28PM +0100, M.-A. Lemburg wrote: > Great :-) If you're the right person to ask, I have a question > about sys.platform on cygwin: if not absolutely necessary, I'd > suggest removing the version number and platform ID (nt, 98) > in sys.platform. > > This information is normally not needed since cygwin tries > to provide a posix subsystem. If it is needed, I'd suggest taking > a look at platform.py (available on my Python Pages) and > adding cygwin support to it. I will respond to the above shortly, but please see below... > > Jason has been monitoring the py-devel list and I try > > to read it over the web on at least a semi-regular basis > > for potential Cygwin related issues. I monitor the distutils list > > > > Feel free to continue to ask Cygwin related questions > > and or request beta testing for Cygwin compatability > > in the future. > > See above ;-) > > I have checked in the patch in a slightly different form. > Could you check whether it now defaults to cygwin for cygwin > compiler Pythons ? Your patch broke the building of the standard (shared extension) modules under Cygwin. For example in 2.1a2, we had the following: .. ./python.exe ./setup.py build running build running build_ext building 'struct' extension creating build creating build/temp.cygwin_nt-4.0-1.3.0-i686-2.1 gcc -g -O2 -Wall -Wstrict-prototypes -DUSE_DL_IMPORT -I. -I/home/jt/src/Python-2.1a2/./Include -IInclude/ -I/usr/local/include -c /home/jt/src/Python-2.1a2/Modules/structmodule.c -o build/temp.cygwin_nt-4.0-1.3.0-i686-2.1/structmodule.o creating build/lib.cygwin_nt-4.0-1.3.0-i686-2.1 gcc -shared -Wl,--enable-auto-image-base build/temp.cygwin_nt-4.0-1.3.0-i686-2.1/structmodule.o -L. -L/usr/local/lib -lpython2.1 -o build/lib.cygwin_nt-4.0-1.3.0-i686-2.1/struct.dll .. Now we have the following: .. PYTHONPATH= ./python.exe ./setup.py build running build running build_ext building 'struct' extension creating build creating build/temp.cygwin_nt-4.0-1.3.0-i686-2.1 gcc -mcygwin -mdll -O -Wall -I. -I/home/jt/src/PythonCvs/./Include -IInclude/ -I/usr/local/include -c /home/jt/src/PythonCvs/Modules/structmodule.c -o build/temp.cygwin_nt-4.0-1.3.0-i686-2.1/structmodule.o writing build/temp.cygwin_nt-4.0-1.3.0-i686-2.1/struct.def creating build/lib.cygwin_nt-4.0-1.3.0-i686-2.1 gcc -mcygwin -mdll -static -s build/temp.cygwin_nt-4.0-1.3.0-i686-2.1/structmodule.o build/temp.cygwin_nt-4.0-1.3.0-i686-2.1/struct.def -L. -L/usr/local/lib -lpython2.1 -o build/lib.cygwin_nt-4.0-1.3.0-i686-2.1/struct.dll /usr/lib/gcc-lib/i686-pc-cygwin/2.95.2-7/../../../../i686-pc-cygwin/bin/ld: cannot find -lpython2.1 collect2: ld returned 1 exit status WARNING: building of extension "struct" failed: command 'gcc' failed with exit status 1 .. During compilation, we now are/have: missing -DUSE_DL_IMPORT option (i.e., CCSHARED variable) extraneous -mcygwin -mdll options During linking, we now are/have: missing -shared -Wl,--enable-auto-image-base options extraneous -mcygwin -mdll -static -s options If I use the latest CVS but downgrade Lib/distutils/ccompiler.py to 1.37, then the build succeeds. I'm willing to help you resolve this problem, please let me know how I can help. Thanks, Jason -- Jason Tishler Director, Software Engineering Phone: +1 (732) 264-8770 x235 Dot Hill Systems Corp. Fax: +1 (732) 264-8798 82 Bethany Road, Suite 7 Email: Jason.Tishler@dothill.com Hazlet, NJ 07730 USA WWW: http://www.dothill.com --futHohkuO/V6j/1Y Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="sysconfig.py.patch" Index: sysconfig.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/distutils/sysconfig.py,v retrieving revision 1.33 diff -u -r1.33 sysconfig.py --- sysconfig.py 2001/02/16 03:31:13 1.33 +++ sysconfig.py 2001/02/20 16:26:45 @@ -117,7 +117,7 @@ 'compiler'. Mainly needed on Unix, so we can plug in the information that varies across Unices and is stored in Python's Makefile. """ - if compiler.compiler_type == "unix": + if compiler.compiler_type in ["unix", "cygwin"]: (cc, opt, ccshared, ldshared, so_ext) = \ get_config_vars('CC', 'OPT', 'CCSHARED', 'LDSHARED', 'SO') --futHohkuO/V6j/1Y-- From thomas.heller@ion-tof.com Wed Feb 21 14:43:02 2001 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Wed Feb 21 14:43:02 2001 Subject: [Distutils] Beta of wininst with uninstaller Message-ID: <002001c09c3e$48359370$e000a8c0@thomasnotebook> This is a multi-part message in MIME format. ------=_NextPart_000_001D_01C09C46.A9DB76F0 Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: 7bit Hi. I've implemented an uninstaller for the bdist_wininst command, the new version is attached to this message for beta testing. Don't bother to read the code, the changes are all in the C-code, which is not included here (but I will post a patch to sourceforge for this). If you want to test it, simply replace your existing distutils/command/bdist_wininst.py file with the attached one. --> Let me know of any problems. <-- What follows are details about the implementation and the design decisions I made: The installer, at install time, creates two additional files in the c:\python20 directory used for uninstalling (let me take distutils-1.0.1 as an example): RemoveDistutils.exe (about 16 kB, which is the actual uninstaller), and Distutils.log (which is a text file, and contains information logged by the installer). Here are the first few lines of Distutils.log: *** Installation started 2001/02/21 20:27 *** Source: C:\sf\distutils\dist\Distutils-1.0.1.win32.exe 020 Reg DB Key: [Software\Microsoft\Windows\CurrentVersion\Uninstall]Distutils-py2.0 040 Reg DB Value: [Software\Microsoft\Windows\CurrentVersion\Uninstall]DisplayName=Python 2.0 Distutils-1.0.2pre 040 Reg DB Value: [Software\Microsoft\Windows\CurrentVersion\Uninstall]UninstallString=C:\Python20\RemoveDistutils.exe -u C:\Python20\Distutils.log 100 Made Dir: C:\Python20\distutils 200 File Copy: C:\Python20\distutils\archive_util.py 200 File Copy: C:\Python20\distutils\bcppcompiler.py 200 File Copy: C:\Python20\distutils\ccompiler.py 200 File Copy: C:\Python20\distutils\cmd.py 100 Made Dir: C:\Python20\distutils\command 200 File Copy: C:\Python20\distutils\command\bdist.py 200 File Copy: C:\Python20\distutils\command\bdist_dumb.py 200 File Copy: C:\Python20\distutils\command\bdist_rpm.py The format is pretty self-explaining, the 3-digit numbers in the first columns simply define an order for removing files, directories and registry entries. Uninstaller information is written to the registry, so that the 'Add/Remove Programs' control panel applet can show (and remove) the program. It appears as 'Python 2.0 Distutils-1.0.1'. The uninstaller reads the logfile, sorts the lines according to the 3-digit numbers, and reverts all these actions starting at the highest codes: Files are removed before directories, registry values are removed before registry keys. Observation: The uninstaller _only_ removes items the installer has created, so, if you did have distutils installed (in the c:\python20\distutils directory) before installing the new version), the 'Made Dir:' entries would not have been written to the logfile because the directories have already been present, and the uninstaller would not remove these directories, even if they would be empty. The 'File Copy:' entries in this case would have been 'File Overwrite:' instead, although the uninstaller would remove these files, because it cannot restore the previously overwritten files. There is one exception: Usually the installer tries to compile the .py files to .pyc and .pyo. Even if this fails, or is disabled in the installer, 'File Copy: xxx.pyc' and 'File Copy: xxx.pyo' entries are written to the logfile, so that they are removed as well. What do you think? Thomas ------=_NextPart_000_001D_01C09C46.A9DB76F0 Content-Type: text/plain; name="bdist_wininst.py" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="bdist_wininst.py" """distutils.command.bdist_wininst Implements the Distutils 'bdist_wininst' command: create a windows installer exe-program.""" # created 2000/06/02, Thomas Heller __revision__ = "$Id: bdist_wininst.py,v 1.19 2001/02/19 09:20:30 theller Exp $" import sys, os, string from distutils.core import Command from distutils.util import get_platform from distutils.dir_util import create_tree, remove_tree from distutils.errors import * class bdist_wininst (Command): description = "create an executable installer for MS Windows" user_options = [('bdist-dir=', 'd', "temporary directory for creating the distribution"), ('keep-temp', 'k', "keep the pseudo-installation tree around after " + "creating the distribution archive"), ('target-version=', 'v', "require a specific python version" + " on the target system"), ('no-target-compile', 'c', "do not compile .py to .pyc on the target system"), ('no-target-optimize', 'o', "do not compile .py to .pyo (optimized)" "on the target system"), ('dist-dir=', 'd', "directory to put final built distributions in"), ('bitmap=', 'b', "bitmap to use for the installer instead of python-powered logo"), ('title=', 't', "title to display on the installer background instead of default"), ] boolean_options = ['keep-temp'] def initialize_options (self): self.bdist_dir = None self.keep_temp = 0 self.no_target_compile = 0 self.no_target_optimize = 0 self.target_version = None self.dist_dir = None self.bitmap = None self.title = None # initialize_options() def finalize_options (self): if self.bdist_dir is None: bdist_base = self.get_finalized_command('bdist').bdist_base self.bdist_dir = os.path.join(bdist_base, 'wininst') if not self.target_version: self.target_version = "" if self.distribution.has_ext_modules(): short_version = sys.version[:3] if self.target_version and self.target_version != short_version: raise DistutilsOptionError, \ "target version can only be" + short_version self.target_version = short_version self.set_undefined_options('bdist', ('dist_dir', 'dist_dir')) # finalize_options() def run (self): if (sys.platform != "win32" and (self.distribution.has_ext_modules() or self.distribution.has_c_libraries())): raise DistutilsPlatformError \ ("distribution contains extensions and/or C libraries; " "must be compiled on a Windows 32 platform") self.run_command('build') install = self.reinitialize_command('install') install.root = self.bdist_dir if os.name != 'nt': # Must force install to use the 'nt' scheme; we set the # prefix too just because it looks silly to put stuff # in (eg.) ".../usr/Scripts", "usr/Include", etc. install.prefix = "Python" install.select_scheme('nt') install_lib = self.reinitialize_command('install_lib') # we do not want to include pyc or pyo files install_lib.compile = 0 install_lib.optimize = 0 install_lib.ensure_finalized() self.announce("installing to %s" % self.bdist_dir) install.ensure_finalized() install.run() # And make an archive relative to the root of the # pseudo-installation tree. fullname = self.distribution.get_fullname() archive_basename = os.path.join(self.bdist_dir, "%s.win32" % fullname) # Our archive MUST be relative to sys.prefix, which is the # same as install_purelib in the 'nt' scheme. root_dir = os.path.normpath(install.install_purelib) # Sanity check: Make sure everything is included for key in ('purelib', 'platlib', 'headers', 'scripts', 'data'): attrname = 'install_' + key install_x = getattr(install, attrname) # (Use normpath so that we can string.find to look for # subdirectories) install_x = os.path.normpath(install_x) if string.find(install_x, root_dir) != 0: raise DistutilsInternalError \ ("'%s' not included in install_lib" % key) arcname = self.make_archive(archive_basename, "zip", root_dir=root_dir) self.create_exe(arcname, fullname, self.bitmap) if not self.keep_temp: remove_tree(self.bdist_dir, self.verbose, self.dry_run) # run() def get_inidata (self): # Return data describing the installation. lines = [] metadata = self.distribution.metadata # Write the [metadata] section. Values are written with # repr()[1:-1], so they do not contain unprintable characters, and # are not surrounded by quote chars. lines.append("[metadata]") # 'info' will be displayed in the installer's dialog box, # describing the items to be installed. info = (metadata.long_description or '') + '\n' for name in dir(metadata): if (name != 'long_description'): data = getattr(metadata, name) if data: info = info + ("\n %s: %s" % \ (string.capitalize(name), data)) lines.append("%s=%s" % (name, repr(data)[1:-1])) # The [setup] section contains entries controlling # the installer runtime. lines.append("\n[Setup]") lines.append("info=%s" % repr(info)[1:-1]) lines.append("target_compile=%d" % (not self.no_target_compile)) lines.append("target_optimize=%d" % (not self.no_target_optimize)) if self.target_version: lines.append("target_version=%s" % self.target_version) title = self.title or self.distribution.get_fullname() lines.append("title=%s" % repr(title)[1:-1]) import time import distutils build_info = "Build %s with distutils-%s" % \ (time.ctime(time.time()), distutils.__version__) lines.append("build_info=%s" % build_info) return string.join(lines, "\n") # get_inidata() def create_exe (self, arcname, fullname, bitmap=None): import struct self.mkpath(self.dist_dir) cfgdata = self.get_inidata() if self.target_version: # if we create an installer for a specific python version, # it's better to include this in the name installer_name = os.path.join(self.dist_dir, "%s.win32-py%s.exe" % (fullname, self.target_version)) else: installer_name = os.path.join(self.dist_dir, "%s.win32.exe" % fullname) self.announce("creating %s" % installer_name) if bitmap: bitmapdata = open(bitmap, "rb").read() bitmaplen = len(bitmapdata) else: bitmaplen = 0 file = open(installer_name, "wb") file.write(self.get_exe_bytes()) if bitmap: file.write(bitmapdata) file.write(cfgdata) header = struct.pack(" Message-ID: <3A94303F.19540BF5@lemburg.com> [New bdist_wininst.py with built-in uninstaller support] I've tested the new bdist_wininst.py with Python 2.0 and so far it works great. I would like to use it for the upcoming release of the eGenix.com mx Extensions 2.0 -- to do this I need your OK though, since the file doesn't include a copyright notice. Thanks for the great work, -- Marc-Andre Lemburg ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Pages: http://www.lemburg.com/python/ From R.Liebscher@gmx.de Thu Feb 22 04:40:47 2001 From: R.Liebscher@gmx.de (Rene Liebscher) Date: Thu Feb 22 04:40:47 2001 Subject: [Distutils] Re: [4suite] Re: Compiling 4Suite 0.10.2beta on Windows References: <20010220123432.F512@dothill.com> <20010221131021.C738@dothill.com> Message-ID: <3A94CCDC.C52F60D3@gmx.de> Jason Tishler wrote: > > John, > > On Tue, Feb 20, 2001 at 07:18:42PM +0000, John J. Lee wrote: > > > > On Tue, 20 Feb 2001, Jason Tishler wrote: > > [...] > > > The current Cygwin gcc release (i.e., 2.95.2-6) still binds against > > > crtdll.dll. Sorry, if my post caused any heartaches. > > > > [...] > > but just to make completely clear: the -c mingw32 option to > > setup.py does appear to work fine at the moment. > > > > You don't need to get mingw32 separately, just get cygwin (unless you > > don't want any of the cygwin stuff, of course). > > If one uses the current Cygwin gcc in Mingw mode (i.e., -mno-cygwin), > then their extensions will use a different C runtime (i.e., crtdll.dll) > than the one used by the Win32 Python interpreter (i.e., msvcrt.dll). > IIRC, this is bound to cause problems such as crashing the interpreter. > For gcc 2.91.57 I had this behaviour, in this case adding the msvcrt library to the to linking libraries solved the problem. A later version (2.95.2-??) worked without problems for the extensions I used. Maybe adding the msvcrt library could also help in your case. (If I understand it right, gcc uses then C functions from msvcrt because it is checked first for the necessary symbols.) Kind regards Rene Liebscher From thomas.heller@ion-tof.com Thu Feb 22 04:41:35 2001 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Thu Feb 22 04:41:35 2001 Subject: [Distutils] Beta of wininst with uninstaller References: <002001c09c3e$48359370$e000a8c0@thomasnotebook> Message-ID: <025701c09cab$326dc810$e000a8c0@thomasnotebook> (Mark Hammond sent these comments directly to me, IMO they should be discussed here). > Hi Thomas, > I am responding to the message I am reading in the archives, hence no > quoting. > > The only thing possibly missing is allowing the package author to have a > custom script for custom installation. If thinking, for example, of COM > servers and such, where the registry entries are _not_ written by the > installer but the product itself, but still need to be removed at uninstall > time. An ever better example is when the package creates extra files (eg, a > cache) that can be removed. > > I may have missed it or an existing facility. Other than that, I think the > work you are doing on this is _excellent_! > No, this functionality is not yet there. I can think of two possibilities: - The WISE (installer) way: Write entries directly into the logfile, parsed and interpreted by the uninstaller. These entries could be given to bdist_wininst on the command-line: setup.py bdist_wininst --log-entry "100 Run Program: package\comserver.py --unregister" - Much more challenging: Let the packager specify a python-script to be run with --install at installation time, and --uninstall at uninstall time. Problem: Can we assume python is still installed at the time the package will be removed? Thomas From thomas.heller@ion-tof.com Thu Feb 22 04:42:48 2001 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Thu Feb 22 04:42:48 2001 Subject: [Distutils] Beta of wininst with uninstaller References: <002001c09c3e$48359370$e000a8c0@thomasnotebook> <3A94303F.19540BF5@lemburg.com> Message-ID: <000f01c09caf$8ec997c0$e000a8c0@thomasnotebook> > [New bdist_wininst.py with built-in uninstaller support] > > I've tested the new bdist_wininst.py with Python 2.0 and so far > it works great. I would like to use it for the upcoming release > of the eGenix.com mx Extensions 2.0 -- to do this I need your > OK though, since the file doesn't include a copyright notice. License: python style Thomas From R.Liebscher@gmx.de Thu Feb 22 04:52:03 2001 From: R.Liebscher@gmx.de (Rene Liebscher) Date: Thu Feb 22 04:52:03 2001 Subject: [Distutils] Cygwin Python Distutils Build Problem References: <20010221140917.F738@dothill.com> Message-ID: <3A94E0DD.1F9F49BA@gmx.de> Jason Tishler wrote: > > On Wed, Feb 21, 2001 at 03:42:04PM +0100, M.-A. Lemburg wrote: > > I don't want distutils to go into the beta with this known > > problem. The patch that you posted seems to be the way to go, > > so I am very much for putting it into the core unless is breaks > > something else which doesn't seem to be the case. > > > > Anyway, please post the patch and a summary of the problem to > > distutils -- maybe Rene knows more about this. > > Marc Lemburg recently committed a patch to facilitate the building of > his mx Extensions under Cygwin. I have attached his patch as the first > attachment for easy access. See the following for details, if interested: > > http://sourceforge.net/patch/?func=detailpatch&patch_id=103868&group_id=5470 > > Unfortunately, this patch breaks the Cygwin Python build. Specifically, > the standard (shared extension) modules no longer build. See the second > attachment for details, if interested. I think the default compiler if the platform is not found in the dictionary, is 'unix'. So it might be better to use "cygwin.*":"unix" instead of "cygwin.*":"cygwin". I wrote cygwinccompiler.py to handle cygwin for MSVC built Pythons, but for a Cygwin built Python it seems more natural to use unixccompiler.py . (The option help describes "unix" as "standard UNIX-style compiler" and for a Cygwin Python and gcc this is the right description. ) > > Being very inexperienced with distutils, I humbly offered the following > WAG patch which corrected the Cygwin Python build problem. The purpose of > this patch is to (re)enable distutils to use the Makefile variables such as > CCSHARED, LDSHARED, etc. when building shared extensions under Cygwin just > like on all other UNIXes. Please see the third attachment for details. > > We have the following questions: > > 1. Is my patch the best way to resolve this issue? It is a possible way, but using unixccompiler is probably better. > 2. The current cygwinccompiler.py (around line 97) has some hardcoded > gcc options (i.e., using -static instead -shared) that appear to > be out of date. Should these be revisited too? Some of these option were needed as workarounds for several bugs in the versions of gcc I tested. (There was last year a version of gcc2.95 which tried to link pythonXX.dll instead libpythonXX.a, this is the reason for the -static option.) We could use the version information to decide if using for an older gcc the hardcoded option or for a newer something generic. But then we should have a way to find out all of the version information, currently you get 2,95,2 for 2.95.2-X. (And at least the first versions of 2.95.2 had some bugs. So how find out X?) > > Hopefully, more knowledgeable people can help us resolve these issues > before the first beta. > > Thanks, > Jason > > > Subject: Re: [Distutils] Beta-release of installers for mx Extensions > Date: Mon, 19 Feb 2001 17:10:03 -0500 > From: Jason Tishler > To: "M.-A. Lemburg" > > Marc, > > On Mon, Feb 19, 2001 at 05:49:28PM +0100, M.-A. Lemburg wrote: > > Great :-) If you're the right person to ask, I have a question > > about sys.platform on cygwin: if not absolutely necessary, I'd > > suggest removing the version number and platform ID (nt, 98) > > in sys.platform. > > > > This information is normally not needed since cygwin tries > > to provide a posix subsystem. If it is needed, I'd suggest taking > > a look at platform.py (available on my Python Pages) and > > adding cygwin support to it. > > I will respond to the above shortly, but please see below... > > > > Jason has been monitoring the py-devel list and I try > > > to read it over the web on at least a semi-regular basis > > > for potential Cygwin related issues. I monitor the distutils list > > > > > > Feel free to continue to ask Cygwin related questions > > > and or request beta testing for Cygwin compatability > > > in the future. > > > > See above ;-) > > > > I have checked in the patch in a slightly different form. > > Could you check whether it now defaults to cygwin for cygwin > > compiler Pythons ? > > Your patch broke the building of the standard (shared extension) modules > under Cygwin. > > For example in 2.1a2, we had the following: > > .. > ./python.exe ./setup.py build > running build > running build_ext > building 'struct' extension > creating build > creating build/temp.cygwin_nt-4.0-1.3.0-i686-2.1 > gcc -g -O2 -Wall -Wstrict-prototypes -DUSE_DL_IMPORT -I. -I/home/jt/src/Python-2.1a2/./Include -IInclude/ -I/usr/local/include -c /home/jt/src/Python-2.1a2/Modules/structmodule.c -o build/temp.cygwin_nt-4.0-1.3.0-i686-2.1/structmodule.o > creating build/lib.cygwin_nt-4.0-1.3.0-i686-2.1 > gcc -shared -Wl,--enable-auto-image-base build/temp.cygwin_nt-4.0-1.3.0-i686-2.1/structmodule.o -L. -L/usr/local/lib -lpython2.1 -o build/lib.cygwin_nt-4.0-1.3.0-i686-2.1/struct.dll > .. > > Now we have the following: > > .. > PYTHONPATH= ./python.exe ./setup.py build > running build > running build_ext > building 'struct' extension > creating build > creating build/temp.cygwin_nt-4.0-1.3.0-i686-2.1 > gcc -mcygwin -mdll -O -Wall -I. -I/home/jt/src/PythonCvs/./Include -IInclude/ -I/usr/local/include -c /home/jt/src/PythonCvs/Modules/structmodule.c -o build/temp.cygwin_nt-4.0-1.3.0-i686-2.1/structmodule.o > writing build/temp.cygwin_nt-4.0-1.3.0-i686-2.1/struct.def > creating build/lib.cygwin_nt-4.0-1.3.0-i686-2.1 > gcc -mcygwin -mdll -static -s build/temp.cygwin_nt-4.0-1.3.0-i686-2.1/structmodule.o build/temp.cygwin_nt-4.0-1.3.0-i686-2.1/struct.def -L. -L/usr/local/lib -lpython2.1 -o build/lib.cygwin_nt-4.0-1.3.0-i686-2.1/struct.dll > /usr/lib/gcc-lib/i686-pc-cygwin/2.95.2-7/../../../../i686-pc-cygwin/bin/ld: cannot find -lpython2.1 > collect2: ld returned 1 exit status > WARNING: building of extension "struct" failed: command 'gcc' failed with exit status 1 > .. > > During compilation, we now are/have: > > missing -DUSE_DL_IMPORT option (i.e., CCSHARED variable) > extraneous -mcygwin -mdll options > > During linking, we now are/have: > > missing -shared -Wl,--enable-auto-image-base options > extraneous -mcygwin -mdll -static -s options > See above why some of these options are there. I also have to confess I am not a specialist with cygwin. If there are other better options (and run also with last years version of gcc) then try it and provide a patch for it. > If I use the latest CVS but downgrade Lib/distutils/ccompiler.py to > 1.37, then the build succeeds. 1.37 is using unixccompiler.py and 1.38 cygwinccompiler.py. > I'm willing to help you resolve this problem, please let me know how I > can help. > > Thanks, > Jason Kind regards Rene Liebscher From mal@lemburg.com Thu Feb 22 05:53:01 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu Feb 22 05:53:01 2001 Subject: [Distutils] Beta of wininst with uninstaller References: <002001c09c3e$48359370$e000a8c0@thomasnotebook> <3A94303F.19540BF5@lemburg.com> <000f01c09caf$8ec997c0$e000a8c0@thomasnotebook> Message-ID: <3A94EF64.9741FFC4@lemburg.com> Thomas Heller wrote: > > > [New bdist_wininst.py with built-in uninstaller support] > > > > I've tested the new bdist_wininst.py with Python 2.0 and so far > > it works great. I would like to use it for the upcoming release > > of the eGenix.com mx Extensions 2.0 -- to do this I need your > > OK though, since the file doesn't include a copyright notice. > License: python style Great :-) BTW, one nit I noted: when two packages depend on each other, the uninstaller fails to remove the directories of the depending package in case the dependent is removed first. Not sure whether one can fix this -- package dependencies are a whole new can of worms... -- Marc-Andre Lemburg ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Pages: http://www.lemburg.com/python/ From mal@lemburg.com Thu Feb 22 06:06:00 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu Feb 22 06:06:00 2001 Subject: [Distutils] Cygwin Python Distutils Build Problem References: <20010221140917.F738@dothill.com> <3A94E0DD.1F9F49BA@gmx.de> Message-ID: <3A94F249.7C0B833@lemburg.com> Rene Liebscher wrote: > > Jason Tishler wrote: > > > > On Wed, Feb 21, 2001 at 03:42:04PM +0100, M.-A. Lemburg wrote: > > > I don't want distutils to go into the beta with this known > > > problem. The patch that you posted seems to be the way to go, > > > so I am very much for putting it into the core unless is breaks > > > something else which doesn't seem to be the case. > > > > > > Anyway, please post the patch and a summary of the problem to > > > distutils -- maybe Rene knows more about this. > > > > Marc Lemburg recently committed a patch to facilitate the building of > > his mx Extensions under Cygwin. I have attached his patch as the first > > attachment for easy access. See the following for details, if interested: > > > > http://sourceforge.net/patch/?func=detailpatch&patch_id=103868&group_id=5470 > > > > Unfortunately, this patch breaks the Cygwin Python build. Specifically, > > the standard (shared extension) modules no longer build. See the second > > attachment for details, if interested. > > I think the default compiler if the platform is not found in the > dictionary, > is 'unix'. > So it might be better to use "cygwin.*":"unix" instead of > "cygwin.*":"cygwin". > I wrote cygwinccompiler.py to handle cygwin for MSVC built Pythons, > but for a Cygwin built Python it seems more natural to use > unixccompiler.py . > > (The option help describes "unix" as "standard UNIX-style compiler" > and for a Cygwin Python and gcc this is the right description. ) If that works, I can fix this easily. Jason ? Note that I'll be more or less offline starting later today (for two weeks), so if this doesn't get settled today, I would appreciate if someone else could make the checkins. Thanks. > > > > Being very inexperienced with distutils, I humbly offered the following > > WAG patch which corrected the Cygwin Python build problem. The purpose of > > this patch is to (re)enable distutils to use the Makefile variables such as > > CCSHARED, LDSHARED, etc. when building shared extensions under Cygwin just > > like on all other UNIXes. Please see the third attachment for details. > > > > We have the following questions: > > > > 1. Is my patch the best way to resolve this issue? > It is a possible way, but using unixccompiler is probably better. > > > 2. The current cygwinccompiler.py (around line 97) has some hardcoded > > gcc options (i.e., using -static instead -shared) that appear to > > be out of date. Should these be revisited too? > Some of these option were needed as workarounds for several bugs in the > versions of gcc I tested. (There was last year a version of gcc2.95 > which > tried to link pythonXX.dll instead libpythonXX.a, this is the reason for > the -static option.) > We could use the version information to decide if using for an older gcc > the hardcoded option or for a newer something generic. > But then we should have a way to find out all of the version > information, > currently you get 2,95,2 for 2.95.2-X. (And at least the first versions > of > 2.95.2 had some bugs. So how find out X?) Oh boy... -- Marc-Andre Lemburg ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Pages: http://www.lemburg.com/python/ From thomas.heller@ion-tof.com Thu Feb 22 08:09:00 2001 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Thu Feb 22 08:09:00 2001 Subject: [Distutils] Beta of wininst with uninstaller References: <002001c09c3e$48359370$e000a8c0@thomasnotebook> <3A94303F.19540BF5@lemburg.com> <000f01c09caf$8ec997c0$e000a8c0@thomasnotebook> <3A94EF64.9741FFC4@lemburg.com> Message-ID: <01fe01c09cd0$78ae66c0$e000a8c0@thomasnotebook> > > BTW, one nit I noted: when two packages depend on each other, > the uninstaller fails to remove the directories of the depending > package in case the dependent is removed first. Not sure whether > one can fix this -- package dependencies are a whole new can of > worms... > I assume this is because the dependent is in the same directory or in a subdirectory as the 'parent' package. I would prefer not to build this into wininst as long as distutils has no solution for it. Thomas From guido@digicool.com Thu Feb 22 08:14:03 2001 From: guido@digicool.com (Guido van Rossum) Date: Thu Feb 22 08:14:03 2001 Subject: [Pycabal] FW: [Distutils] Python Siphon name change please? In-Reply-To: Your message of "Thu, 22 Feb 2001 01:45:14 EST." References: Message-ID: <200102221311.IAA15371@cj20424-a.reston1.va.home.com> > Guido van Rossum writes > >Sorry for the introsion. I just found out that there's a thing called > >Python-Siphon being discussed on the distutils-sig (to which I'm not > >subscribed), which is CPAN functionality. I believe that falls under > >the Catalog SIG's charter too (to which I *am* subscribed, I believe > >-- but it's been dead since last November), so I'mm ccing there. > > > >I love the concept. But please don't use that name. I searched > >Google for "python siphon" and found out it is a technical term for > >something used to siphon water into or out of aquariums (?). That's > >cute, but makes it hard to find through web searches, and that's not > >what we want for a CPAN thingie! :-) > > > >Feel free to flame me if this is inappropriate! > > > >--Guido van Rossum (home page: http://www.python.org/~guido/) Robin Becker replied: > Well there was a very long discussion about this on clp, but I guess > dictators don't listen to ordinary folk. I guess the common fate of > dictators is that the masses eventually catch up with them in surprising > ways. Well, I guess I'm getting what I asked for! I did said "please" though. :-) > Siphon seems quite a good name in terms of the etymology and rhyme. If > all new decisions on module name have to be referred to google it'll be > a bit limiting. Sure. I thought this was going to be more than a module though? > Feel free to post an alternate and then we can submit it to google. It seems you've already made up your mind, so I'll back off now. Just one question: I wonder why did this never came up on the catalog-sig. Did we fail to advertise the SIG when it was created? How can we prevent such mistakes in the future? --Guido van Rossum (home page: http://www.python.org/~guido/) From mal@lemburg.com Thu Feb 22 08:57:01 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu Feb 22 08:57:01 2001 Subject: [Distutils] Beta of wininst with uninstaller References: <002001c09c3e$48359370$e000a8c0@thomasnotebook> <3A94303F.19540BF5@lemburg.com> <000f01c09caf$8ec997c0$e000a8c0@thomasnotebook> <3A94EF64.9741FFC4@lemburg.com> <01fe01c09cd0$78ae66c0$e000a8c0@thomasnotebook> Message-ID: <3A951A62.4B7AF76A@lemburg.com> Thomas Heller wrote: > > > > > BTW, one nit I noted: when two packages depend on each other, > > the uninstaller fails to remove the directories of the depending > > package in case the dependent is removed first. Not sure whether > > one can fix this -- package dependencies are a whole new can of > > worms... > > > I assume this is because the dependent is in the same directory > or in a subdirectory as the 'parent' package. > > I would prefer not to build this into wininst as long as > distutils has no solution for it. Right. -- Marc-Andre Lemburg ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Pages: http://www.lemburg.com/python/ From Jason.Tishler@dothill.com Thu Feb 22 09:40:01 2001 From: Jason.Tishler@dothill.com (Jason Tishler) Date: Thu Feb 22 09:40:01 2001 Subject: [Distutils] Cygwin Python Distutils Build Problem In-Reply-To: <3A94F249.7C0B833@lemburg.com>; from mal@lemburg.com on Thu, Feb 22, 2001 at 12:04:41PM +0100 References: <20010221140917.F738@dothill.com> <3A94E0DD.1F9F49BA@gmx.de> <3A94F249.7C0B833@lemburg.com> Message-ID: <20010222093859.J915@dothill.com> Rene, On Thu, Feb 22, 2001 at 12:04:41PM +0100, M.-A. Lemburg wrote: > Rene Liebscher wrote: > > I think the default compiler if the platform is not found in the > > dictionary, > > is 'unix'. > > So it might be better to use "cygwin.*":"unix" instead of > > "cygwin.*":"cygwin". > > I wrote cygwinccompiler.py to handle cygwin for MSVC built Pythons, > > but for a Cygwin built Python it seems more natural to use > > unixccompiler.py . > > > > (The option help describes "unix" as "standard UNIX-style compiler" > > and for a Cygwin Python and gcc this is the right description. ) > > If that works, I can fix this easily. Jason ? Given the information above, I feel that your solution is the best. It restores the Cygwin Python build back to what it was before Marc's patch to ccompiler.py. Additionally, by obviating my proposed patch to sysconfig.py, this will prevent affecting cygwinccompiler.py's behavior especially with regards to building extensions for MSVC Python. This had always concerned me, hence, requiring me to label my proposed patch as a WAG. > Note that I'll be more or less offline starting later today (for two > weeks), so if this doesn't get settled today, I would appreciate if > someone else could make the checkins. Thanks. Since Marc may already be gone and offline for two weeks, would you be willing to check in this change? Do you want me to submit a patch to SF? Thanks, Jason -- Jason Tishler Director, Software Engineering Phone: +1 (732) 264-8770 x235 Dot Hill Systems Corp. Fax: +1 (732) 264-8798 82 Bethany Road, Suite 7 Email: Jason.Tishler@dothill.com Hazlet, NJ 07730 USA WWW: http://www.dothill.com From Jason.Tishler@dothill.com Thu Feb 22 10:43:01 2001 From: Jason.Tishler@dothill.com (Jason Tishler) Date: Thu Feb 22 10:43:01 2001 Subject: [Distutils] Cygwin Python Distutils Build Problem In-Reply-To: <3A94E0DD.1F9F49BA@gmx.de>; from R.Liebscher@gmx.de on Thu, Feb 22, 2001 at 10:50:21AM +0100 References: <20010221140917.F738@dothill.com> <3A94E0DD.1F9F49BA@gmx.de> Message-ID: <20010222104233.K915@dothill.com> Rene, On Thu, Feb 22, 2001 at 10:50:21AM +0100, Rene Liebscher wrote: > > I think the default compiler if the platform is not found in the > > dictionary, > > is 'unix'. > > So it might be better to use "cygwin.*":"unix" instead of > > "cygwin.*":"cygwin". One downside to the above or put another way one good thing about using cygwinccompiler.py instead of unixccompiler.py for Cygwin is that .def files were automatically created. This could be helpful with building shared extensions that were originally created (and only concerned with running) on UNIX and are missing the DL_EXPORT macro when defining their initialization function. I had to submit numerous patches to SF to correct nits like this. Is there a good way to "blend" the cygwinccompiler.py and unixccompiler.py functionality? > > 2. The current cygwinccompiler.py (around line 97) has some hardcoded > > gcc options (i.e., using -static instead -shared) that appear to > > be out of date. Should these be revisited too? > Some of these option were needed as workarounds for several bugs in the > versions of gcc I tested. (There was last year a version of gcc2.95 > which > tried to link pythonXX.dll instead libpythonXX.a, this is the reason for > the -static option.) > We could use the version information to decide if using for an older gcc > the hardcoded option or for a newer something generic. > But then we should have a way to find out all of the version > information, > currently you get 2,95,2 for 2.95.2-X. (And at least the first versions > of > 2.95.2 had some bugs. So how find out X?) Older Cygwin versions didn't indicate the "-X" in the "--version" output, but the newer ones do. For example, $ gcc --version 2.95.2-6 So may be cygwinccompiler.py could key off of this information with the following algorithm: If the version of gcc is 2.95.2-6 or greater, then use the new options; otherwise, use the old ones. Note that the new options still need to be determined... > > During compilation, we now are/have: > > > > missing -DUSE_DL_IMPORT option (i.e., CCSHARED variable) > > extraneous -mcygwin -mdll options > > > > During linking, we now are/have: > > > > missing -shared -Wl,--enable-auto-image-base options > > extraneous -mcygwin -mdll -static -s options > > > See above why some of these options are there. I also have to confess I > am not > a specialist with cygwin. If there are other better options (and run > also with > last years version of gcc) then try it and provide a patch for it. OK, but I'm not that motivated (at the moment) because I don't normally build MSVC extensions. If you do get motivated, then I'll try to see if there are better options to use. > > If I use the latest CVS but downgrade Lib/distutils/ccompiler.py to > > 1.37, then the build succeeds. > 1.37 is using unixccompiler.py and 1.38 cygwinccompiler.py. I know, the point of the above was to clearly demonstrate to Marc that his latest patch to ccompiler.py was causing the problem. Thanks, Jason -- Jason Tishler Director, Software Engineering Phone: +1 (732) 264-8770 x235 Dot Hill Systems Corp. Fax: +1 (732) 264-8798 82 Bethany Road, Suite 7 Email: Jason.Tishler@dothill.com Hazlet, NJ 07730 USA WWW: http://www.dothill.com From Jason.Tishler@dothill.com Thu Feb 22 11:25:00 2001 From: Jason.Tishler@dothill.com (Jason Tishler) Date: Thu Feb 22 11:25:00 2001 Subject: [Distutils] Re: [4suite] Re: Compiling 4Suite 0.10.2beta on Windows In-Reply-To: <3A94CCDC.C52F60D3@gmx.de>; from R.Liebscher@gmx.de on Thu, Feb 22, 2001 at 09:25:00AM +0100 References: <20010220123432.F512@dothill.com> <20010221131021.C738@dothill.com> <3A94CCDC.C52F60D3@gmx.de> Message-ID: <20010222112330.L915@dothill.com> Rene, On Thu, Feb 22, 2001 at 09:25:00AM +0100, Rene Liebscher wrote: > Jason Tishler wrote: > > If one uses the current Cygwin gcc in Mingw mode (i.e., -mno-cygwin), > > then their extensions will use a different C runtime (i.e., crtdll.dll) > > than the one used by the Win32 Python interpreter (i.e., msvcrt.dll). > > IIRC, this is bound to cause problems such as crashing the interpreter. I was just referring to the following above: http://starship.python.net/crew/kernr/mingw32/Notes.html Specifically, the following snippet: Caveats and Miscellaneous Notes o Always use MSVCRT.dll: The official Win32 Python distribution is linked against MSVCRT.dll. Extension modules compiled against CRTDLL.dll work in many instances, but will cause crashes sometimes. For example, when a FILE * pointer is passed from the extension to any function from python15.dll, the interpreter will crash. Similarly, whenever the interpreter tries to print the representation of an extension object, it will also crash: >>> import myextension >>> foo = myextension.new_object() >>> foo *** Horrible, disgusting crash. *** > For gcc 2.91.57 I had this behaviour, in this case adding the msvcrt > library to the to linking libraries solved the problem. > A later version (2.95.2-??) worked without problems for the extensions > I used. > Maybe adding the msvcrt library could also help in your case. > (If I understand it right, gcc uses then C functions from msvcrt because > it is checked first for the necessary symbols.) I never thought about manually adding the msvcrt.lib -- that is an interesting solution. The bottom line is if you do an "objdump -p" on your shared extension and it is *not* dependent on crtdll.dll, then it will function properly. If it is, then it may not. Jason -- Jason Tishler Director, Software Engineering Phone: +1 (732) 264-8770 x235 Dot Hill Systems Corp. Fax: +1 (732) 264-8798 82 Bethany Road, Suite 7 Email: Jason.Tishler@dothill.com Hazlet, NJ 07730 USA WWW: http://www.dothill.com From robin@jessikat.fsnet.co.uk Thu Feb 22 18:25:01 2001 From: robin@jessikat.fsnet.co.uk (Robin Becker) Date: Thu Feb 22 18:25:01 2001 Subject: [Distutils] Python Siphon name change please? In-Reply-To: <20010221100503.B29605@ute.cnri.reston.va.us> References: <200102210300.WAA08543@cj20424-a.reston1.va.home.com> <20010221100503.B29605@ute.cnri.reston.va.us> Message-ID: In article <20010221100503.B29605@ute.cnri.reston.va.us>, Andrew Kuchling writes >On Wed, Feb 21, 2001 at 09:11:52AM +0000, Robin Becker wrote: >>Siphon seems quite a good name in terms of the etymology and rhyme. If >>all new decisions on module name have to be referred to google it'll be >>a bit limiting. > >And frankly, quibbling about the name when we don't actually have an >implementation is premature! > >--amk in the absence of alternate proposals I supose the Siphon has it :) -- Robin Becker From dubois@users.sourceforge.net Thu Feb 22 23:51:01 2001 From: dubois@users.sourceforge.net (Paul F. Dubois) Date: Thu Feb 22 23:51:01 2001 Subject: [Distutils] Odd thought In-Reply-To: Message-ID: Right now if you get a package on a Windows machine you can't double-click setup.py and have anything good happen. But, if setup.py were setup.ipy you could bind a python "%1" install command to the suffix .ipy so this would indeed work. The only thing required to make this happen for free is a change to the Windows installers to recognize the extension. It wouldn't have any particular bad effect on Unix. You either do ./setup.ipy or python setup.ipy install, both work. Or am I being weird? Mnemonic: I py with my little eye, a Python extension to install. From R.Liebscher@gmx.de Fri Feb 23 03:57:01 2001 From: R.Liebscher@gmx.de (Rene Liebscher) Date: Fri Feb 23 03:57:01 2001 Subject: [Distutils] Odd thought References: Message-ID: <3A96259A.4DDB1CF@gmx.de> "Paul F. Dubois" wrote: > > Right now if you get a package on a Windows machine you can't double-click > setup.py and have anything good happen. But, if setup.py were setup.ipy you > could bind a python "%1" install command to the suffix .ipy so this would > indeed work. The only thing required to make this happen for free is a > change to the Windows installers to recognize the extension. It wouldn't > have any particular bad effect on Unix. You either do ./setup.ipy or python > setup.ipy install, both work. > > Or am I being weird? Mnemonic: I py with my little eye, a Python extension > to install. > Why not change the "py" binding to "python %1"? If anyone writes a console program then it is probably intended to be run with arguments. (If it is started from the console window as "python foo.py" the "python %1" binding doesn't hurt.) (And for non-console programs one would call this file "foo.pyw".) The only problem which remains is that the extension "pyw" is only used on Windows. But if we could tag such non-console programs in distutil's setup.py then distutil's install could rename them from "py" to "pyw" when it runs on Windows. (Or from "pyw" to "py" for all non-Windows platforms, then the extension would be the tag.) Kind regards Rene Liebscher From R.Liebscher@gmx.de Fri Feb 23 04:11:01 2001 From: R.Liebscher@gmx.de (Rene Liebscher) Date: Fri Feb 23 04:11:01 2001 Subject: [Distutils] Odd thought References: <3A96259A.4DDB1CF@gmx.de> Message-ID: <3A9628E4.B5F1559B@gmx.de> Rene Liebscher wrote: > > ... > Why not change the "py" binding to "python %1"? ==>correction cmd /k "python %1" Somehow I missed these two words. Kind regards Rene Liebscher From jack@oratrix.nl Fri Feb 23 06:06:02 2001 From: jack@oratrix.nl (Jack Jansen) Date: Fri Feb 23 06:06:02 2001 Subject: [Distutils] Odd thought In-Reply-To: Message by Rene Liebscher , Fri, 23 Feb 2001 09:55:54 +0100 , <3A96259A.4DDB1CF@gmx.de> Message-ID: <20010223110539.CBC603C2032@snelboot.oratrix.nl> There may be another solution to the double-clicking-setup.py problem. As there isn't a command line at all on the Mac I had to create a GetArgv() dialog that posts a dialog with options to the user and creates a command line. The good news is that the distutils interface to getargs is *very* good: you have all possible arguments available with helptexts, etc. So if someone ported this dialog to Windows you'd be all set: if setup.py is called without arguments you simply post the dialog. -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm From thomas.heller@ion-tof.com Fri Feb 23 08:02:02 2001 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Fri Feb 23 08:02:02 2001 Subject: [Distutils] Odd thought References: <20010223110539.CBC603C2032@snelboot.oratrix.nl> Message-ID: <022201c09d98$a8588fe0$e000a8c0@thomasnotebook> [Jack Jansen] > There may be another solution to the double-clicking-setup.py problem. > > As there isn't a command line at all on the Mac I had to create a GetArgv() > dialog that posts a dialog with options to the user and creates a command line. > > The good news is that the distutils interface to getargs is *very* good: you > have all possible arguments available with helptexts, etc. > > So if someone ported this dialog to Windows you'd be all set: if setup.py is > called without arguments you simply post the dialog. The problem I see is that there is no way to create a dialog-box on windows without either requiring win32all to be installed, or to use a custom python interpreter 'py_dialog.exe' which would then display a dialog box (and maybe also display the output of the script) Thomas From guido@digicool.com Fri Feb 23 08:54:01 2001 From: guido@digicool.com (Guido van Rossum) Date: Fri Feb 23 08:54:01 2001 Subject: [Distutils] Odd thought In-Reply-To: Your message of "Fri, 23 Feb 2001 12:05:39 +0100." <20010223110539.CBC603C2032@snelboot.oratrix.nl> References: <20010223110539.CBC603C2032@snelboot.oratrix.nl> Message-ID: <200102231351.IAA23683@cj20424-a.reston1.va.home.com> > There may be another solution to the double-clicking-setup.py problem. > > As there isn't a command line at all on the Mac I had to create a GetArgv() > dialog that posts a dialog with options to the user and creates a command line. > > The good news is that the distutils interface to getargs is *very* good: you > have all possible arguments available with helptexts, etc. > > So if someone ported this dialog to Windows you'd be all set: if setup.py is > called without arguments you simply post the dialog. That sounds to me like the right solution. --Guido van Rossum (home page: http://www.python.org/~guido/) From R.Liebscher@gmx.de Fri Feb 23 08:59:00 2001 From: R.Liebscher@gmx.de (Rene Liebscher) Date: Fri Feb 23 08:59:00 2001 Subject: [Distutils] Odd thought References: <20010223110539.CBC603C2032@snelboot.oratrix.nl> <022201c09d98$a8588fe0$e000a8c0@thomasnotebook> Message-ID: <3A966C31.D8B8A142@gmx.de> Thomas Heller wrote: > > [Jack Jansen] > > There may be another solution to the double-clicking-setup.py problem. > > > > As there isn't a command line at all on the Mac I had to create a GetArgv() > > dialog that posts a dialog with options to the user and creates a command line. > > > > The good news is that the distutils interface to getargs is *very* good: you > > have all possible arguments available with helptexts, etc. > > > > So if someone ported this dialog to Windows you'd be all set: if setup.py is > > called without arguments you simply post the dialog. > > The problem I see is that there is no way to create a dialog-box > on windows without either requiring win32all to be installed, > or to use a custom python interpreter 'py_dialog.exe' which > would then display a dialog box (and maybe also display the > output of the script) I think most people have Tk installed with their Python, so it is probably not that big problem. Rene From guido@digicool.com Fri Feb 23 09:03:00 2001 From: guido@digicool.com (Guido van Rossum) Date: Fri Feb 23 09:03:00 2001 Subject: [Distutils] Odd thought In-Reply-To: Your message of "Fri, 23 Feb 2001 14:00:54 +0100." <022201c09d98$a8588fe0$e000a8c0@thomasnotebook> References: <20010223110539.CBC603C2032@snelboot.oratrix.nl> <022201c09d98$a8588fe0$e000a8c0@thomasnotebook> Message-ID: <200102231400.JAA23859@cj20424-a.reston1.va.home.com> > The problem I see is that there is no way to create a dialog-box > on windows without either requiring win32all to be installed, > or to use a custom python interpreter 'py_dialog.exe' which > would then display a dialog box (and maybe also display the > output of the script) You could at least *try* to import Tkinter and see if it exists. --Guido van Rossum (home page: http://www.python.org/~guido/) From robin@jessikat.fsnet.co.uk Fri Feb 23 10:01:01 2001 From: robin@jessikat.fsnet.co.uk (Robin Becker) Date: Fri Feb 23 10:01:01 2001 Subject: [Distutils] Odd thought In-Reply-To: <022201c09d98$a8588fe0$e000a8c0@thomasnotebook> References: <20010223110539.CBC603C2032@snelboot.oratrix.nl> <022201c09d98$a8588fe0$e000a8c0@thomasnotebook> Message-ID: In article <022201c09d98$a8588fe0$e000a8c0@thomasnotebook>, Thomas Heller writes >[Jack Jansen] >> There may be another solution to the double-clicking-setup.py problem. >> >> As there isn't a command line at all on the Mac I had to create a GetArgv() >> dialog that posts a dialog with options to the user and creates a command >line. >> >> The good news is that the distutils interface to getargs is *very* good: you >> have all possible arguments available with helptexts, etc. >> >> So if someone ported this dialog to Windows you'd be all set: if setup.py is >> called without arguments you simply post the dialog. > >The problem I see is that there is no way to create a dialog-box >on windows without either requiring win32all to be installed, >or to use a custom python interpreter 'py_dialog.exe' which >would then display a dialog box (and maybe also display the >output of the script) > >Thomas That's not quite true; with Sam Rushing's calldll extension you could get some dialogs built without too large an addition, but that stuff is a bit hacky. -- Robin Becker From paul@pfdubois.com Fri Feb 23 10:04:00 2001 From: paul@pfdubois.com (Paul F. Dubois) Date: Fri Feb 23 10:04:00 2001 Subject: [Distutils] Odd thought In-Reply-To: <200102231400.JAA23859@cj20424-a.reston1.va.home.com> Message-ID: Aha. If a setup.py started with this: import sys if len(sys.argv) == 1: sys.argv.append('install') it would work to double-click it. No window needed. (:-> This is no more offensive than my ipy idea. For those who don't like this, it could print a prompt "Install? (y/n): ", read the answer, etc. That said, if distutils' setup command contained this code... -----Original Message----- From: guido@cj20424-a.reston1.va.home.com [mailto:guido@cj20424-a.reston1.va.home.com]On Behalf Of Guido van Rossum Sent: Friday, February 23, 2001 6:00 AM To: Thomas Heller Cc: Rene Liebscher; Jack Jansen; Paul F. Dubois; distutils-sig@python.org; David Ascher Subject: Re: [Distutils] Odd thought > The problem I see is that there is no way to create a dialog-box > on windows without either requiring win32all to be installed, > or to use a custom python interpreter 'py_dialog.exe' which > would then display a dialog box (and maybe also display the > output of the script) You could at least *try* to import Tkinter and see if it exists. --Guido van Rossum (home page: http://www.python.org/~guido/) From R.Liebscher@gmx.de Fri Feb 23 10:21:00 2001 From: R.Liebscher@gmx.de (Rene Liebscher) Date: Fri Feb 23 10:21:00 2001 Subject: [Distutils] Odd thought References: Message-ID: <3A967F6F.3142FA84@gmx.de> "Paul F. Dubois" wrote: > > Aha. If a setup.py started with this: > import sys > if len(sys.argv) == 1: > sys.argv.append('install') > > it would work to double-click it. No window needed. (:-> But what if an user tries on the command line "python setup.py" normally he gets an usage description and doesn't start "install". May be the changing of the "py" binding to "cmd /k python %1" is the best solution here. If you double-click on the file, it opens a console window, runs the file with python, DOESN'T close the window so you can see the result and you are already in the right directory to make another try with arguments. (And it works for all command line python programs, not only distutils setup.py.) > > This is no more offensive than my ipy idea. For those who don't like this, > it could print a prompt "Install? (y/n): ", read the answer, etc. > > That said, if distutils' setup command contained this code... > Rene From jack@oratrix.nl Fri Feb 23 11:02:09 2001 From: jack@oratrix.nl (Jack Jansen) Date: Fri Feb 23 11:02:09 2001 Subject: [Distutils] Odd thought In-Reply-To: Message by "Paul F. Dubois" , Fri, 23 Feb 2001 06:56:03 -0800 , Message-ID: <20010223160141.5126A3C203D@snelboot.oratrix.nl> > Aha. If a setup.py started with this: > import sys > if len(sys.argv) == 1: > sys.argv.append('install') > > it would work to double-click it. No window needed. (:-> Or you could go into "interactive mode": show the help message, as is done now, and then say "Command: ", read a line, strip, push into sys.argv. Actually this could be API-compatible with the dialog suggestion I made earlier. The code would just try a couple of the dialogs (Win32GetArgv, MacGetArgv, TkinterGetArgv, StdinGetArgv) until it finds one that can be imported. -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm From akuchlin@mems-exchange.org Fri Feb 23 11:05:15 2001 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Fri Feb 23 11:05:15 2001 Subject: [Distutils] Odd thought In-Reply-To: <20010223160141.5126A3C203D@snelboot.oratrix.nl>; from jack@oratrix.nl on Fri, Feb 23, 2001 at 05:01:40PM +0100 References: <20010223160141.5126A3C203D@snelboot.oratrix.nl> Message-ID: <20010223110455.E2879@ute.cnri.reston.va.us> On Fri, Feb 23, 2001 at 05:01:40PM +0100, Jack Jansen wrote: >Or you could go into "interactive mode": show the help message, as is done >now, and then say "Command: ", read a line, strip, push into sys.argv. Interesting idea. However, is this the job of the Distutils or of some third-party package handled by the Catalog-SIG? Consider Perl's CPAN.pm as an example; it's a shell for searching for modules, fetching them, and installing them. I don't know if it handles uninstalling, too. --amk From thomas.heller@ion-tof.com Fri Feb 23 11:17:00 2001 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Fri Feb 23 11:17:00 2001 Subject: [Distutils] Odd thought References: <20010223110539.CBC603C2032@snelboot.oratrix.nl><022201c09d98$a8588fe0$e000a8c0@thomasnotebook> Message-ID: <036801c09db4$097d3ad0$e000a8c0@thomasnotebook> > That's not quite true; with Sam Rushing's calldll extension you could > get some dialogs built without too large an addition, but that stuff is > a bit hacky. Yes, I know (and I also did such a calldll, even before I knew about python, for other languages). To be honest: I find the calldll approach less hacky than tkinter (formatting commands into strings, and sending them to another language for interpretation) This may also explain why I didn't think of Tkinter... Thomas From guido@digicool.com Fri Feb 23 11:23:01 2001 From: guido@digicool.com (Guido van Rossum) Date: Fri Feb 23 11:23:01 2001 Subject: [Distutils] Odd thought In-Reply-To: Your message of "Fri, 23 Feb 2001 17:16:54 +0100." <036801c09db4$097d3ad0$e000a8c0@thomasnotebook> References: <20010223110539.CBC603C2032@snelboot.oratrix.nl><022201c09d98$a8588fe0$e000a8c0@thomasnotebook> <036801c09db4$097d3ad0$e000a8c0@thomasnotebook> Message-ID: <200102231620.LAA25220@cj20424-a.reston1.va.home.com> > Yes, I know (and I also did such a calldll, even before I knew > about python, for other languages). > > To be honest: I find the calldll approach less hacky > than tkinter (formatting commands into strings, and > sending them to another language for interpretation) But is calldll a standard part of Python distributions? Tcl/Tk is! --Guido van Rossum (home page: http://www.python.org/~guido/) From thomas.heller@ion-tof.com Fri Feb 23 11:33:12 2001 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Fri Feb 23 11:33:12 2001 Subject: [Distutils] Odd thought References: <20010223110539.CBC603C2032@snelboot.oratrix.nl><022201c09d98$a8588fe0$e000a8c0@thomasnotebook> <036801c09db4$097d3ad0$e000a8c0@thomasnotebook> <200102231620.LAA25220@cj20424-a.reston1.va.home.com> Message-ID: <03aa01c09db6$44feeca0$e000a8c0@thomasnotebook> > > Yes, I know (and I also did such a calldll, even before I knew > > about python, for other languages). > > > > To be honest: I find the calldll approach less hacky > > than tkinter (formatting commands into strings, and > > sending them to another language for interpretation) > > But is calldll a standard part of Python distributions? Tcl/Tk is! Is this also true for ActiveState's distro? Thomas From thomas.heller@ion-tof.com Fri Feb 23 11:37:02 2001 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Fri Feb 23 11:37:02 2001 Subject: [Distutils] distutils, uninstaller Message-ID: <03f201c09db6$cf201990$e000a8c0@thomasnotebook> I've uploaded the bdist_wininst uninstaller patch to sourceforge: http://sourceforge.net/patch/?func=detailpatch&patch_id=103948&group_id=5470 Just in case someone cares. Another thing: Shouldn't the distutils version number change before the beta? I suggest going from 1.0.1 to 1.0.2. Thomas Heller From guido@digicool.com Fri Feb 23 11:41:06 2001 From: guido@digicool.com (Guido van Rossum) Date: Fri Feb 23 11:41:06 2001 Subject: [Distutils] Odd thought In-Reply-To: Your message of "Fri, 23 Feb 2001 17:32:52 +0100." <03aa01c09db6$44feeca0$e000a8c0@thomasnotebook> References: <20010223110539.CBC603C2032@snelboot.oratrix.nl><022201c09d98$a8588fe0$e000a8c0@thomasnotebook> <036801c09db4$097d3ad0$e000a8c0@thomasnotebook> <200102231620.LAA25220@cj20424-a.reston1.va.home.com> <03aa01c09db6$44feeca0$e000a8c0@thomasnotebook> Message-ID: <200102231638.LAA25348@cj20424-a.reston1.va.home.com> > > But is calldll a standard part of Python distributions? Tcl/Tk is! > Is this also true for ActiveState's distro? Currently not. I've griped to them abot this, but they don't seem to like Tcl/Tk much. Now that they've offered support for Tcl/Tk, maybe it's time to try again... --Guido van Rossum (home page: http://www.python.org/~guido/) From paulp@ActiveState.com Fri Feb 23 15:11:11 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Fri Feb 23 15:11:11 2001 Subject: [Distutils] Odd thought References: <20010223110539.CBC603C2032@snelboot.oratrix.nl><022201c09d98$a8588fe0$e000a8c0@thomasnotebook> <036801c09db4$097d3ad0$e000a8c0@thomasnotebook> <200102231620.LAA25220@cj20424-a.reston1.va.home.com> <03aa01c09db6$44feeca0$e000a8c0@thomasnotebook> <200102231638.LAA25348@cj20424-a.reston1.va.home.com> Message-ID: <3A96C460.59876A43@ActiveState.com> Guido van Rossum wrote: > > > > But is calldll a standard part of Python distributions? Tcl/Tk is! > > Is this also true for ActiveState's distro? > > Currently not. I've griped to them abot this, but they don't seem to > like Tcl/Tk much. Now that they've offered support for Tcl/Tk, maybe > it's time to try again... A soon-to-be-released version of ActivePython will have Tk as an installation option. Calldll will also likely be an option. -- Vote for Your Favorite Python & Perl Programming Accomplishments in the first Active Awards! http://www.ActiveState.com/Awards From akuchlin@mems-exchange.org Fri Feb 23 18:10:08 2001 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Fri Feb 23 18:10:08 2001 Subject: [Distutils] distutils, uninstaller In-Reply-To: <03f201c09db6$cf201990$e000a8c0@thomasnotebook>; from thomas.heller@ion-tof.com on Fri, Feb 23, 2001 at 05:36:44PM +0100 References: <03f201c09db6$cf201990$e000a8c0@thomasnotebook> Message-ID: <20010223180937.A5178@ute.cnri.reston.va.us> On Fri, Feb 23, 2001 at 05:36:44PM +0100, Thomas Heller wrote: >I've uploaded the bdist_wininst uninstaller >patch to sourceforge: >http://sourceforge.net/patch/?func=detailpatch&patch_id=103948&group_id=5470 Can anyone take a look at the patch just as a sanity check? I can't really comment on it, but if someone else gives it a look, Thomas can go ahead and check it in. >Another thing: Shouldn't the distutils version number change >before the beta? I suggest going from 1.0.1 to 1.0.2. Good point. It doesn't look like beta1 will be happening until late next week due to the nested scoping changes, but I'll do that before the release. --amk From nhv@cape.com Sun Feb 25 18:18:06 2001 From: nhv@cape.com (Norman Vine) Date: Sun Feb 25 18:18:06 2001 Subject: [Distutils] RE: [wxPython] wxPython & Cygwin In-Reply-To: <000001c09f4c$4823a3c0$a300a8c0@nhv> Message-ID: <000001c09f7f$feb96600$a300a8c0@nhv> Hi All, I have gotten the majority of wxPython 2.2.5 to run with Cygwin 1.1.8 :-)) This required a fair bit of 'coercion' to the distutils cygcompiler.py module, I used the wxPython my_distutils.py as a reference, I also had to add an additional section in setup.py for the cygwin compiler I will be glad to supply the specifics to anyone interested. FYI I overcame the 'resource file not found" problem I mentioned in my earlier post by linking the wxWindows resource file into the python executable. IMO this is not really satisfactory but seems to work. Cheers Norman Vine From jafo@tummy.com Mon Feb 26 03:40:10 2001 From: jafo@tummy.com (Sean Reifschneider) Date: Mon Feb 26 03:40:10 2001 Subject: [Distutils] Re: CPAN functionality for python In-Reply-To: <20010215170227.K23577@tummy.com>; from jafo-pythonlist@tummy.com on Thu, Feb 15, 2001 at 05:02:27PM -0700 References: <14984.49127.853606.286789@w221.z064000254.bwi-md.dsl.cnc.net> <20010215170227.K23577@tummy.com> Message-ID: <20010226013903.B3221@tummy.com> Unfortunately, this week has been pretty busy. I've been giving the idea some thought again (it's been something I've wanted to work on for quite a while), and just finally today had the chance to look at pythonsiphon. I'm not really clear (from the code what) as to what it's job is to be. It smells like it's wanting to be something that understands, given a distutils package, how to install it. My thoughts on that are that it's a job that's probably going to be something subject largely to being handed off to appropriate scripts based on the platform and some user input (for example, a user may prefer to have packages they download installed in ~/lib/python1.5, instead of /usr/lib/python1.5). Ideally though, the tool should be able to deal with allowing the user to select their preferred distribution media. I'd prefer to see an ix86 RedHat 7.0 RPM, a SRPM, and then would fall back to a distutils file or tar file. As far as the server side for allowing users to query packages, dependencies, and locations... I had actually hoped to pound out some code on this, but that just didn't happen. I have built up a schema which I think encompasses everything I see such a system wanting to track. Initial versions probably won't track all that, but I believe they should be considered. The schema is attached below. A quick overview of the tables. I hate the name "items", but it's the best I could come up with. That's what I'm calling the general module (things that can be imported) or package (for example, programs like Sketch -- why not have them in here as well) description. In this case, let's use "sockserv". The URL would point to the main location you can find "sockserv" module information. "Packages" are actually meant to describe what you can download. This database will track multiple versions, include a checksum and information on what platform it's meant for (for binary distributions, for example). Mirrors are handled by a special table. The idea is that one should be able to add mirrors without having to list all the items in that particular mirror (again). New entries should be able to be listed without having to list all the mirrors they might reside on. So, I've come up with the idea that a mirror simply specifies a "prefix" for the URL listed in the "locations" table. To be honest, my plan is to have a special-case "null" mirror which non-mirror members are listed under -- that would have an empty prefix. So, given a package, version, format, architecture, etc... The "locations" table will produce list of all locations which that file can be found at. Note that there is a dependency table which lists that one "item" relies on another "item". As far as the network interface goes, at the current moment I'm thinking that queries will be submitted to a web server CGI as a POST, and the results will be returned as the body as text/plain... I'll probably use netstrings (as discussed before) to encode things for transport. So, any comments? Sean ========================= #ISSUES: # This doesn't handle partial mirrors. Should it? How? CREATE TABLE items ( id serial primary key, lastUpdated datetime default 'now', name text, description text, type text, # module, package homepageURL text, ) CREATE TABLE packages ( id serial primary key, itemID int4, version text, md5sum text, filesize text, insertedOn datetime default 'now', description text, # description about this specific file platformName text, platformVersion text, architecture text, packageFormat text, ) # a collection of URLs that all contain the same set of files (within some # precision). A mirror can be added without having to add all the URLs it # contains. Not useful for handling partial-mirrors though. CREATE TABLE mirrors ( id serial primary key, name text, baseURL text, ) CREATE TABLE locations ( id serial primary key, packageID int4, mirrorID int4, # if found on a mirror, list it URL text, # append this text to the mirror baseURL ) CREATE TABLE depends ( id serial primary key, itemID int4, requiresItemID int4 ) CREATE TABLE users ( id serial primary key, name text, password text, lastLogin datetime default NULL, loginCookie text ) CREATE TABLE maintainers ( userID int4, type text, # package, item piID int4 ) CREATE TABLE ranking ( id serial primary key, rank int2, itemID int4, userID int4 date datetime default 'now', ) -- Millions long for immortality who don't know what to do with themselves on a rainy Sunday afternoon. -- Heinlein Sean Reifschneider, Inimitably Superfluous tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python From bsass@freenet.edmonton.ab.ca Mon Feb 26 04:57:08 2001 From: bsass@freenet.edmonton.ab.ca (Bruce Sass) Date: Mon Feb 26 04:57:08 2001 Subject: [Distutils] Re: CPAN functionality for python In-Reply-To: <20010226013903.B3221@tummy.com> Message-ID: On Mon, 26 Feb 2001, Sean Reifschneider wrote: <...> > My thoughts on that are that it's a job that's probably going to be > something subject largely to being handed off to appropriate scripts > based on the platform and some user input (for example, a user may > prefer to have packages they download installed in ~/lib/python1.5, > instead of /usr/lib/python1.5). > > Ideally though, the tool should be able to deal with allowing the user > to select their preferred distribution media. I'd prefer to see an ix86 > RedHat 7.0 RPM, a SRPM, and then would fall back to a distutils file or > tar file. Do you mean: A tool that would download a python pkg, then pass it onto scripts that could turn it into a .deb, .rpm (bunch of flavours), tarball, etc., then install it. or A tool that would download pre-packaged distro specific pkgs. Regarding the first... I like this because users will always get an installable package that fits into their system, and every system can have packages in their native format - something you can't expect if developers are doing the distro specific packaging and arbitrary individuals are filling in the rest. I realize the distros will probably start auto-building once python modules are available in a standard format, but users should also have the ability to do the packaging with a minimum of hoops to jump through. As for the second... it seems like an awful waste of archive space to have the same code in many different formats, just because some want to put stuff in /usr/share and others in /usr/lib, etc. Maybe I've just been spoiled by the Debian packaging system and tools, but tarballs/zips and third-party packages are definately second class, I would much rather have the packages created on the fly then installed by a script (probably provided by a Debian developer). I suppose you could replace "Debian" with RH, Mandrake, Microsoft, ... whatever your fav distro is. - Bruce From doughellmann@home.com Mon Feb 26 06:51:07 2001 From: doughellmann@home.com (Doug Hellmann) Date: Mon Feb 26 06:51:07 2001 Subject: [Distutils] Re: CPAN functionality for python In-Reply-To: References: Message-ID: <01022606501904.15046@branagan> On Mon, 26 Feb 2001, Bruce Sass wrote: > On Mon, 26 Feb 2001, Sean Reifschneider wrote: > <...> > > My thoughts on that are that it's a job that's probably going to be > > something subject largely to being handed off to appropriate scripts > > based on the platform and some user input (for example, a user may > > prefer to have packages they download installed in ~/lib/python1.5, > > instead of /usr/lib/python1.5). > > > > Ideally though, the tool should be able to deal with allowing the user > > to select their preferred distribution media. I'd prefer to see an ix86 > > RedHat 7.0 RPM, a SRPM, and then would fall back to a distutils file or > > tar file. > > Do you mean: > > A tool that would download a python pkg, then pass it onto scripts > that could turn it into a .deb, .rpm (bunch of flavours), tarball, > etc., then install it. > > or > > A tool that would download pre-packaged distro specific pkgs. > > > Regarding the first... I like this because users will always get an > installable package that fits into their system, and every system can > have packages in their native format - something you can't expect if > developers are doing the distro specific packaging and arbitrary > individuals are filling in the rest. I realize the distros will > probably start auto-building once python modules are available in a > standard format, but users should also have the ability to do the > packaging with a minimum of hoops to jump through. Why is this a desirable feature? Is it worth the complexity? What if Python kept up with its own packages and modules like XEmacs does? We could then use the same format on all platforms -- it might even be something we roll ourself to make sure we can unpack it with *only* Python installed on a platform. Doug From bsass@freenet.edmonton.ab.ca Mon Feb 26 13:18:01 2001 From: bsass@freenet.edmonton.ab.ca (Bruce Sass) Date: Mon Feb 26 13:18:01 2001 Subject: [Distutils] Re: CPAN functionality for python In-Reply-To: <01022606501904.15046@branagan> Message-ID: On Mon, 26 Feb 2001, Doug Hellmann wrote: > On Mon, 26 Feb 2001, Bruce Sass wrote: > > Do you mean: > > > > A tool that would download a python pkg, then pass it onto scripts > > that could turn it into a .deb, .rpm (bunch of flavours), tarball, > > etc., then install it. > > > > or > > > > A tool that would download pre-packaged distro specific pkgs. > > > > > > Regarding the first... I like this because users will always get an > > installable package that fits into their system, and every system can > > have packages in their native format - something you can't expect if > > developers are doing the distro specific packaging and arbitrary > > individuals are filling in the rest. I realize the distros will > > probably start auto-building once python modules are available in a > > standard format, but users should also have the ability to do the > > packaging with a minimum of hoops to jump through. > > Why is this a desirable feature? Is it worth the complexity? Because distros are not always building what users are running. It should be no more complex than defining the inter-dependencies between the modules in the archive, and declaring what is required from outside the archive. The translation from a generic pkg to a distro specific pkg would be a problem for the disto to solve (probably a matter of repacking info about the pkg, then using whatever packaging tools they have). A second hook would be needed for mapping the python pkg name to the distro specific name. > What if Python kept up with its own packages and modules like XEmacs does? We > could then use the same format on all platforms -- it might even be something > we roll ourself to make sure we can unpack it with *only* Python installed on a > platform. There are enough different packaging schemes around that it seems unnecessarily heavy-handed to pick one (or two or three) over the others - Python should not appear to favor any paticular system. The only way I can see to do that is to not package to anyones spec, but provide enough information so that anyone can package to their fav spec. - Bruce From guido@digicool.com Mon Feb 26 13:26:01 2001 From: guido@digicool.com (Guido van Rossum) Date: Mon Feb 26 13:26:01 2001 Subject: [Distutils] Re: CPAN functionality for python In-Reply-To: Your message of "Mon, 26 Feb 2001 11:14:35 MST." References: Message-ID: <200102261822.NAA26272@cj20424-a.reston1.va.home.com> > There are enough different packaging schemes around that it seems > unnecessarily heavy-handed to pick one (or two or three) over the > others - Python should not appear to favor any paticular system. The > only way I can see to do that is to not package to anyones spec, but > provide enough information so that anyone can package to their > fav spec. But practicality beats purity: if a large fraction of users, especially newbie users, are using a specific platform, it makes a lot of sense to provide pre-packaged distributions for them! E.g. Windows installers, Red Hat RPMs. (So far my experience with Windows installers is much more positive than with RPMs though -- RPMs forever seem to depend on some version of some other RPM that you don't have.) --Guido van Rossum (home page: http://www.python.org/~guido/) From phrxy@csv.warwick.ac.uk Mon Feb 26 13:58:08 2001 From: phrxy@csv.warwick.ac.uk (John J. Lee) Date: Mon Feb 26 13:58:08 2001 Subject: [Distutils] Re: CPAN functionality for python In-Reply-To: <200102261822.NAA26272@cj20424-a.reston1.va.home.com> Message-ID: On Mon, 26 Feb 2001, Guido van Rossum wrote: [...] > installers, Red Hat RPMs. (So far my experience with Windows > installers is much more positive than with RPMs though -- RPMs forever > seem to depend on some version of some other RPM that you don't have.) Surely this is a function of free vs. non-free software as opposed to which particular packaging format you are using? John From bsass@freenet.edmonton.ab.ca Mon Feb 26 14:00:02 2001 From: bsass@freenet.edmonton.ab.ca (Bruce Sass) Date: Mon Feb 26 14:00:02 2001 Subject: [Distutils] Re: CPAN functionality for python In-Reply-To: <200102261822.NAA26272@cj20424-a.reston1.va.home.com> Message-ID: On Mon, 26 Feb 2001, Guido van Rossum wrote: > > There are enough different packaging schemes around that it seems > > unnecessarily heavy-handed to pick one (or two or three) over the > > others - Python should not appear to favor any paticular system. The > > only way I can see to do that is to not package to anyones spec, but > > provide enough information so that anyone can package to their > > fav spec. > > But practicality beats purity: if a large fraction of users, > especially newbie users, are using a specific platform, it makes a lot > of sense to provide pre-packaged distributions for them! E.g. Windows > installers, Red Hat RPMs. (So far my experience with Windows > installers is much more positive than with RPMs though -- RPMs forever > seem to depend on some version of some other RPM that you don't have.) Sure, but if you can build packages on the fly, transparently to the user... it is a "six of one, half dozen of the other" situation. I would expect there to be situation where autobuilding would fail or not be practical (C based modules), supporting "binary" packages would be a necessity. Ya, RPMs are not known for doing a good job of dependencies. Debian would be a better model to look at with respect to what dependency info is required and how it should be handled. I rarely (can't remember the last time) have dependency problems with DEBs, that includes third party DEBs (they usually have problems with "Policy", but that's a whole other story ;). - Bruce From tim.one@home.com Mon Feb 26 14:15:09 2001 From: tim.one@home.com (Tim Peters) Date: Mon Feb 26 14:15:09 2001 Subject: [Distutils] Re: CPAN functionality for python In-Reply-To: Message-ID: [Guido] > (So far my experience with Windows installers is much more > positive than with RPMs though -- RPMs forever seem to depend > on some version of some other RPM that you don't have.) [John J. Lee] > Surely this is a function of free vs. non-free software as opposed to > which particular packaging format you are using? I doubt it has much to do with either. It's more that, over very painful years of installers that routinely crashed, Windows apps that rely on anything external have learned to ship with *everything* they need. Heck, the Windows Python installer even ships with a copy of Microsoft's C runtime libraries, just in case the platform installed on doesn't have a reasonably up-to-date copy. If we did something similar on Linux, we'd ship Python there with its own glibc <0.7 wink>. shudder-if-you-like-but-it-would-have-stopped-the-platform- LONG_BIT-screwups-ly y'rs - tim From bsass@freenet.edmonton.ab.ca Mon Feb 26 14:25:18 2001 From: bsass@freenet.edmonton.ab.ca (Bruce Sass) Date: Mon Feb 26 14:25:18 2001 Subject: [Distutils] Re: CPAN functionality for python In-Reply-To: Message-ID: On Mon, 26 Feb 2001, John J. Lee wrote: > On Mon, 26 Feb 2001, Guido van Rossum wrote: > [...] > > installers, Red Hat RPMs. (So far my experience with Windows > > installers is much more positive than with RPMs though -- RPMs forever > > seem to depend on some version of some other RPM that you don't have.) > > Surely this is a function of free vs. non-free software as opposed to > which particular packaging format you are using? I think the problem with RPMs is that they depend on a specific package, instead of depending on the contents of a package. e.g., If you have code depending on libfoo-1.x, depending on the package that provides libfoo-1.1 will work, but is no good to people with libfoo-1.2, the dependency should be on any package that provides libfoo-1. - Bruce From skip@mojam.com (Skip Montanaro) Mon Feb 26 15:05:09 2001 From: skip@mojam.com (Skip Montanaro) (Skip Montanaro) Date: Mon Feb 26 15:05:09 2001 Subject: [Distutils] Re: CPAN functionality for python In-Reply-To: <200102261822.NAA26272@cj20424-a.reston1.va.home.com> References: <200102261822.NAA26272@cj20424-a.reston1.va.home.com> Message-ID: <15002.46773.642780.730206@beluga.mojam.com> Guido> (So far my experience with Windows installers is much more Guido> positive than with RPMs though -- RPMs forever seem to depend on Guido> some version of some other RPM that you don't have.) Guido, Can you elaborate? I rarely have any problem installing stuff that's packaged as an RPM. If I understand your statement correctly, the Windows Python installer had the same sort of problem with Tcl/Tk until the decision was made to bite the bullet and simply include it as part of the installer. -- Skip Montanaro (skip@mojam.com) Support Mojam & Musi-Cal: http://www.musi-cal.com/sponsor.shtml (847)971-7098 From mwa@gate.net Mon Feb 26 15:08:09 2001 From: mwa@gate.net (Mark W. Alexander) Date: Mon Feb 26 15:08:09 2001 Subject: [Distutils] Re: CPAN functionality for python In-Reply-To: <200102261822.NAA26272@cj20424-a.reston1.va.home.com> Message-ID: On Mon, 26 Feb 2001, Guido van Rossum wrote: > Date: Mon, 26 Feb 2001 13:22:45 -0500 > From: Guido van Rossum > To: Bruce Sass > Cc: python-list@python.org, distutils-sig@python.org > Subject: Re: [Distutils] Re: CPAN functionality for python > > But practicality beats purity: if a large fraction of users, > especially newbie users, are using a specific platform, it makes a lot > of sense to provide pre-packaged distributions for them! E.g. Windows > installers, Red Hat RPMs. (So far my experience with Windows > installers is much more positive than with RPMs though -- RPMs forever > seem to depend on some version of some other RPM that you don't have.) > > --Guido van Rossum (home page: http://www.python.org/~guido/) Isn't this the point of distutils --bdist functionality? The catalog should provide download capability of any binary format supported byu distutils. It can either be done on the fly at request time, or batched periodically depending on the space availability of the catalog providing site. Mark mwa@gate.net From tim.one@home.com Mon Feb 26 15:22:09 2001 From: tim.one@home.com (Tim Peters) Date: Mon Feb 26 15:22:09 2001 Subject: [Distutils] Re: CPAN functionality for python In-Reply-To: <15002.46773.642780.730206@beluga.mojam.com> Message-ID: [Skip Montanaro, to Guido] > Can you elaborate? I rarely have any problem installing stuff that's > packaged as an RPM. If I understand your statement correctly, the > Windows Python installer had the same sort of problem with Tcl/Tk until > the decision was made to bite the bullet and simply include it as part > of the installer. Actually, the Python Windows installer included the Tcl/Tk installer for a long time without solving anything. It wasn't until Python *stopped* running the Tcl/Tk installer, and took the *pieces* of Tcl/Tk Python needs and installed them itself, that the endless Windows problems went away. Python no longer tries to cooperate with any pre-existing Tcl/Tk installation on Windows, and indeed does all it can to ensure that the Tcl/Tk it installs neither affects nor is affected by any other Tcl/Tk installation on the box, or that may exist in the future, or that may get uninstalled in the future. removing-potential-cooperation-served-in-reality-to-remove-potential- conflict-ly y'rs - tim From robin@alldunn.com Mon Feb 26 15:39:09 2001 From: robin@alldunn.com (Robin Dunn) Date: Mon Feb 26 15:39:09 2001 Subject: [Distutils] Re: CPAN functionality for python References: Message-ID: <02f501c0a033$d7bedd00$0100a8c0@Rogue> > > > > But practicality beats purity: if a large fraction of users, > > especially newbie users, are using a specific platform, it makes a lot > > of sense to provide pre-packaged distributions for them! E.g. Windows > > installers, Red Hat RPMs. (So far my experience with Windows > > installers is much more positive than with RPMs though -- RPMs forever > > seem to depend on some version of some other RPM that you don't have.) > > > > --Guido van Rossum (home page: http://www.python.org/~guido/) > > Isn't this the point of distutils --bdist functionality? The > catalog should provide download capability of any binary format > supported byu distutils. It can either be done on the fly at > request time, or batched periodically depending on the space > availability of the catalog providing site. > That would work for pure python packages, but not for any that contains an extension module. They would still have to be built on the target platform and then the taget package format can be built with --bdist. -- Robin Dunn Software Craftsman robin@AllDunn.com Java give you jitters? http://wxPython.org Relax with wxPython! From amos@digicool.com Mon Feb 26 18:23:01 2001 From: amos@digicool.com (Amos Latteier) Date: Mon Feb 26 18:23:01 2001 Subject: [Distutils] [Announce] Distutils Catalog Prototype Message-ID: <3A9AE2C4.37CDFF22@digicool.com> Hi Guys, I finally got off my butt and built a distutils catalog prototype. Now for a limited time you can get to it at http://63.230.174.230:8080/ This is a Zope site that allows you to upload distutils archives and search them. I added a basic HTML and XML interface. The XML interface should be used by command-line clients. The user name and password for uploading are both 'guest'. I am going to remove this account in a couple days since it is fundamentally insecure. I mostly want to post this now so folks can see what I'm doing. If you want to play with the prototype more, get Zope 2.3 and install the DistutilsArchive20010226.tgz file. I've preloaded the site with one archive (distutils 1.0.1). Try out the site by uploading your own archives. If the site can't parse them take a look in Parse.py and figure out what the problem is and send me a patch. Issues * Executing setup.py to find meta-data is insecure. Plus I've found that many existing distutils archives don't actually expose the meta-data very well this way. I propose that we fix this by having distutils write a meta-data file when you build an archive. * What API would command-line tools like? XML in some home-cooked DTD? Maybe XML-RPC? Let me know what you think! -Amos -- Amos Latteier mailto:amos@digicool.com Digital Creations http://www.digicool.com From robin@alldunn.com Mon Feb 26 18:56:10 2001 From: robin@alldunn.com (Robin Dunn) Date: Mon Feb 26 18:56:10 2001 Subject: [Distutils] Re: [Catalog-sig] [Announce] Distutils Catalog Prototype References: <3A9AE2C4.37CDFF22@digicool.com> Message-ID: <03c901c0a04f$66ea3720$0100a8c0@Rogue> > Hi Guys, > > I finally got off my butt and built a distutils catalog prototype. Now > for a limited time you can get to it at > > http://63.230.174.230:8080/ > > This is a Zope site that allows you to upload distutils archives and > search them. I added a basic HTML and XML interface. The XML interface > should be used by command-line clients. The user name and password for > uploading are both 'guest'. I am going to remove this account in a > couple days since it is fundamentally insecure. I mostly want to post > this now so folks can see what I'm doing. If you want to play with the > prototype more, get Zope 2.3 and install the > DistutilsArchive20010226.tgz file. > > I've preloaded the site with one archive (distutils 1.0.1). Try out the > site by uploading your own archives. If the site can't parse them take a > look in Parse.py and figure out what the problem is and send me a patch. > > Issues > > * Executing setup.py to find meta-data is insecure. Plus I've found > that many existing distutils archives don't actually expose the > meta-data very well this way. I propose that we fix this by having > distutils write a meta-data file when you build an archive. This sounds like a good idea. It would probably be useful for other things as well. > * What API would command-line tools like? XML in some home-cooked DTD? > Maybe XML-RPC? Definitily XML-RPC. It's so easy to use from the client side that it doesn't make sense to NOT have it IMHO. That isn't to say that some other format in addition to XML-RPC shouldn't be done too, both may make sense. > > Let me know what you think! > I don't have time to debug it right now (maybe later) but if you try to load this file: http://download.sourceforge.net/pybsddb/bsddb3-3.0b3.tar.gz then you get this error: Zope Error Zope has encountered an error while publishing this resource. Error Type: ParseError Error Value: Archive does not contain a setup.py file ... Traceback (innermost last): File /home/amos/DistSite/lib/python/ZPublisher/Publish.py, line 223, in publish_module File /home/amos/DistSite/lib/python/ZPublisher/Publish.py, line 187, in publish File /home/amos/DistSite/lib/python/Zope/__init__.py, line 221, in zpublisher_exception_hook File /home/amos/DistSite/lib/python/ZPublisher/Publish.py, line 171, in publish File /home/amos/DistSite/lib/python/ZPublisher/mapply.py, line 160, in mapply (Object: addMethod) File /home/amos/DistSite/lib/python/ZPublisher/Publish.py, line 112, in call_object (Object: addMethod) File /home/amos/DistSite/lib/python/Products/DistutilsArchive/Archive.py, line 25, in addMethod File /home/amos/DistSite/lib/python/Products/DistutilsArchive/Archive.py, line 84, in update (Object: RoleManager) File /home/amos/DistSite/lib/python/Products/DistutilsArchive/Parse.py, line 63, in parse_meta_data ParseError: (see above) The archive does have a setup.py. -- Robin Dunn Software Craftsman robin@AllDunn.com Java give you jitters? http://wxPython.org Relax with wxPython! From jafo@tummy.com Mon Feb 26 20:10:10 2001 From: jafo@tummy.com (Sean Reifschneider) Date: Mon Feb 26 20:10:10 2001 Subject: [Distutils] Re: CPAN functionality for python In-Reply-To: ; from bsass@freenet.edmonton.ab.ca on Mon, Feb 26, 2001 at 11:14:35AM -0700 References: <01022606501904.15046@branagan> Message-ID: <20010226180935.B7531@tummy.com> On Mon, Feb 26, 2001 at 11:14:35AM -0700, Bruce Sass wrote: >from outside the archive. The translation from a generic pkg to a >distro specific pkg would be a problem for the disto to solve >(probably a matter of repacking info about the pkg, then using The only way having automaticly-generated packages be possible is if we really force a conformance from the authors of the packages. I've packaged quite a number of RPMs in my day, and you can get pretty close to just wrapping a tar file with some meta-information in some cases, but these are the minority. In most cases you have to provide patches which fix incorrect paths in the tar file (for example, the Python tar file uses "#!/usr/local/bin/python" all over the place). If we can force the distributions to be more compatible with being packaged, it may be very easy. Otherwise, we may have to rely on people contributing their time to building RPMs if we want to have RPMs. >only way I can see to do that is to not package to anyones spec, but >provide enough information so that anyone can package to their >fav spec. I don't agree... I believe that we should embrace the different packaging formats, not act like they don't exist. If the distutils package can be made to generate native RPMs, that's great. The *CLIENT* program should dictate that. If it can handle turning a distutil package into an RPM, it'll have it's preference be a distutil package. I don't believe having the archive not able to deal with different package formats is acceptable. If that were the case, then if the package didn't provide enough information to build an RPM, the user would have no option within the archive of working around it. Sean -- When in doubt, use brute force. -- Ken Thompson Sean Reifschneider, Inimitably Superfluous tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python From jafo@tummy.com Mon Feb 26 20:27:09 2001 From: jafo@tummy.com (Sean Reifschneider) Date: Mon Feb 26 20:27:09 2001 Subject: [Distutils] Re: CPAN functionality for python In-Reply-To: ; from bsass@freenet.edmonton.ab.ca on Mon, Feb 26, 2001 at 02:56:51AM -0700 References: <20010226013903.B3221@tummy.com> Message-ID: <20010226182601.D7531@tummy.com> On Mon, Feb 26, 2001 at 02:56:51AM -0700, Bruce Sass wrote: >Do you mean: Ideally, it would be able to deal with both. The ability to download a distutils package and produce a .deb or a .rpm is more a function of the client program than of the archive network... It also largely depends on distutils as well. >As for the second... it seems like an awful waste of archive space to >have the same code in many different formats, just because some want >to put stuff in /usr/share and others in /usr/lib, etc. I'm really not familiar with .deb packages. Howeer, for RPMs, if the author has included a .spec file in the tar file, you can simply run "rpm -ta foo.tar.gz" and the results will (should) be a compiled binary RPM. That is, if the author of the package has thought ahead about producing an RPM. The question becomes: Is it reasonable to require authors of packages for the archive network to do the same? Convince me it is... Sean -- "McGuyver stole all his tricks from Dr. Who." Sean Reifschneider, Inimitably Superfluous tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python From jafo@tummy.com Mon Feb 26 20:32:01 2001 From: jafo@tummy.com (Sean Reifschneider) Date: Mon Feb 26 20:32:01 2001 Subject: [Distutils] Re: CPAN functionality for python In-Reply-To: ; from bsass@freenet.edmonton.ab.ca on Mon, Feb 26, 2001 at 11:57:36AM -0700 References: <200102261822.NAA26272@cj20424-a.reston1.va.home.com> Message-ID: <20010226183138.E7531@tummy.com> On Mon, Feb 26, 2001 at 11:57:36AM -0700, Bruce Sass wrote: >Ya, RPMs are not known for doing a good job of dependencies. Debian They aren't? The example given was a situation where one could say that RPM was doing *TOO* good a job of dependencies -- it was failing to install because you didn't have what was needed. For quite a long time I was runing a 5.2 machine running the newest versions of GTK, but older versions of glibc. Practicly none of the stock binary RPMs would install on it because of that. However, if you pick up an SRPM, and do "rpm --rebuild package.src.rpm", it will build the RPM against *EXACTLY* the set of libraries and files you have. It works brilliantly. Sean -- "No early worm is giving ME the bird!" -- Bullwinkle J. Moose Sean Reifschneider, Inimitably Superfluous tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python From jafo@tummy.com Mon Feb 26 20:37:00 2001 From: jafo@tummy.com (Sean Reifschneider) Date: Mon Feb 26 20:37:00 2001 Subject: [Distutils] Re: CPAN functionality for python In-Reply-To: <01022606501904.15046@branagan>; from doughellmann@home.com on Mon, Feb 26, 2001 at 06:45:33AM -0500 References: <01022606501904.15046@branagan> Message-ID: <20010226183606.F7531@tummy.com> On Mon, Feb 26, 2001 at 06:45:33AM -0500, Doug Hellmann wrote: >What if Python kept up with its own packages and modules like XEmacs does? We >could then use the same format on all platforms -- it might even be something >we roll ourself to make sure we can unpack it with *only* Python installed on a >platform. My understanding is that that's what distutils is meant to do, to an extent. The problem is that it's not entirely unreasonable to expect that an "rpm -qa" produce a list of all the software that's installed on your box. If you don't use an RPM, you can't really make use of that, and packages WILL get dropped after a re-install. Sean -- ISA isn't dead, it's just that people wish it were. The correct term for this condition is "legacy"... -- Sean Reifschneider, 1999 Sean Reifschneider, Inimitably Superfluous tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python From jafo@tummy.com Mon Feb 26 20:40:09 2001 From: jafo@tummy.com (Sean Reifschneider) Date: Mon Feb 26 20:40:09 2001 Subject: [Distutils] Re: CPAN functionality for python In-Reply-To: ; from bsass@freenet.edmonton.ab.ca on Mon, Feb 26, 2001 at 12:22:47PM -0700 References: Message-ID: <20010226183857.G7531@tummy.com> On Mon, Feb 26, 2001 at 12:22:47PM -0700, Bruce Sass wrote: >I think the problem with RPMs is that they depend on a specific >package, instead of depending on the contents of a package. In RPM, you can let the packaging system decide what revisions of various packages are needed (by looking at the results, it will find out what libraries it links against, what's in the she-bang (#!) lines, etc). That mechanism is usually the most paranoid about versions (if we linked against glibc 2.1.7.9.pi, then that's exactly what we expect). On the other hand, if you know better, as an application developer, you can include something like: Requires: python >= 1.5.2 in the .spec file which says "I know that 1.5.1 won't work, but I hope that 1.6 won't break this package". Sean -- Some girl with psychic power, She said "T-bone, what's your sign?" I blinked and answered "Neon", I thought I'd blow her mind. Sean Reifschneider, Inimitably Superfluous tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python From amos@digicool.com Mon Feb 26 20:48:01 2001 From: amos@digicool.com (Amos Latteier) Date: Mon Feb 26 20:48:01 2001 Subject: [Distutils] Re: [Catalog-sig] [Announce] Distutils Catalog Prototype References: <3A9AE2C4.37CDFF22@digicool.com> <03c901c0a04f$66ea3720$0100a8c0@Rogue> Message-ID: <3A9B04D8.B81DED2B@digicool.com> Robin Dunn wrote: > > * Executing setup.py to find meta-data is insecure. Plus I've found > > that many existing distutils archives don't actually expose the > > meta-data very well this way. I propose that we fix this by having > > distutils write a meta-data file when you build an archive. > > This sounds like a good idea. It would probably be useful for other things > as well. Once we get agreement that a meta-data file is needed then we can begin the horrific task if figuring out what format it should be in ;-) > > * What API would command-line tools like? XML in some home-cooked DTD? > > Maybe XML-RPC? > > Definitily XML-RPC. It's so easy to use from the client side that it > doesn't make sense to NOT have it IMHO. That isn't to say that some other > format in addition to XML-RPC shouldn't be done too, both may make sense. OK, I'll try to put together something with XML-RPC. > I don't have time to debug it right now (maybe later) but if you try to load > this file: > http://download.sourceforge.net/pybsddb/bsddb3-3.0b3.tar.gz Hmm. I get a different error: Error Type: RuntimeError Error Value: 'distutils.core.setup()' was never called -- perhaps '/var/tmp/@11142.3/bsddb3-3.0b3/setup.py' is not a Distutils setup script? ... File /home/amos/DistSite/lib/python/Products/DistutilsArchive/Parse.py, line 69, in parse_meta_data File /usr/local/lib/python1.5/site-packages/distutils/core.py, line 221, in run_setup RuntimeError: (see above) Sure enough, when I fire up the interpreter and try to manually use distutils.core.run_setup on your archive I get the same error. But your setup.py looks good. My guess is that this is a distutils error... -Amos -- Amos Latteier mailto:amos@digicool.com Digital Creations http://www.digicool.com From amos@digicool.com Mon Feb 26 20:57:01 2001 From: amos@digicool.com (Amos Latteier) Date: Mon Feb 26 20:57:01 2001 Subject: [Distutils] Re: [Catalog-sig] [Announce] Distutils Catalog Prototype References: <3A9AE2C4.37CDFF22@digicool.com> <03c901c0a04f$66ea3720$0100a8c0@Rogue> Message-ID: <3A9B06E2.F782BB52@digicool.com> Robin Dunn wrote: > I don't have time to debug it right now (maybe later) but if you try to load > this file: > http://download.sourceforge.net/pybsddb/bsddb3-3.0b3.tar.gz OK, the problem is that I don't have berkeley db installed on my server. Therefor setup.py chokes with Can't find a local db3 installation. This keeps distutils.core.run_setup from being able to extract the meta-data from the archive. This is yet another reason that we need distutils to write meta-data to a file when building an archive. Extracting meta-data by running setup.py may fail for many reasons. -Amos -- Amos Latteier mailto:amos@digicool.com Digital Creations http://www.digicool.com From robin@alldunn.com Mon Feb 26 20:59:01 2001 From: robin@alldunn.com (Robin Dunn) Date: Mon Feb 26 20:59:01 2001 Subject: [Distutils] Re: [Catalog-sig] [Announce] Distutils Catalog Prototype References: <3A9AE2C4.37CDFF22@digicool.com> <03c901c0a04f$66ea3720$0100a8c0@Rogue> <3A9B04D8.B81DED2B@digicool.com> Message-ID: <041a01c0a060$a36d0770$0100a8c0@Rogue> > Robin Dunn wrote: > > > > * Executing setup.py to find meta-data is insecure. Plus I've found > > > that many existing distutils archives don't actually expose the > > > meta-data very well this way. I propose that we fix this by having > > > distutils write a meta-data file when you build an archive. > > > > This sounds like a good idea. It would probably be useful for other things > > as well. > > Once we get agreement that a meta-data file is needed then we can begin > the horrific task if figuring out what format it should be in ;-) Well as long as there's a utility function in distutils somewhere to read it and populate a dictionary or an instance of some DistMetaData class then it doesn't matter what format it's stored in, right? ;-) > Error Type: RuntimeError > Error Value: 'distutils.core.setup()' was never called -- perhaps > '/var/tmp/@11142.3/bsddb3-3.0b3/setup.py' is not a Distutils setup > script? > RuntimeError: (see above) > > Sure enough, when I fire up the interpreter and try to manually use > distutils.core.run_setup on your archive I get the same error. But your > setup.py looks good. My guess is that this is a distutils error... > It does a sys.exit if it can't find a BerkeleyDB library. I imagine that is not a very good thing to do when trying to extract meta data this way... -- Robin Dunn Software Craftsman robin@AllDunn.com Java give you jitters? http://wxPython.org Relax with wxPython! From doughellmann@home.com Mon Feb 26 23:18:10 2001 From: doughellmann@home.com (Doug Hellmann) Date: Mon Feb 26 23:18:10 2001 Subject: [Distutils] Re: CPAN functionality for python - requirements In-Reply-To: <20010226183606.F7531@tummy.com> References: <01022606501904.15046@branagan> <20010226183606.F7531@tummy.com> Message-ID: <01022623140707.15046@branagan> On Mon, 26 Feb 2001, Sean Reifschneider wrote: > On Mon, Feb 26, 2001 at 06:45:33AM -0500, Doug Hellmann wrote: > >What if Python kept up with its own packages and modules like XEmacs does? We > >could then use the same format on all platforms -- it might even be something > >we roll ourself to make sure we can unpack it with *only* Python installed on a > >platform. > > My understanding is that that's what distutils is meant to do, to an extent. > The problem is that it's not entirely unreasonable to expect that an > "rpm -qa" produce a list of all the software that's installed on your > box. If you don't use an RPM, you can't really make use of that, and > packages WILL get dropped after a re-install. The command "rpm -qa" only produces meaningful results on a system where RPM is the default and primary package installation system. There are more OSes where that is *not* the case (and where RPM isn't installed at all) than where it is. It still isn't clear to me why we would want to use more than one distribution package format. It appears that one argument is so that sysadmins can use those tools to track what is installed on the system. But isn't that part of what this new thing (cyphon?) is supposed to do? Do we want Python to require an external package management system on each platform where this tool is supported, or do we want a Python-esq tool which just works the way it is supposed to, in the same way, everywhere Python is available? But I think we're arguing over a design before we've worked out requirements. The basic tasks of this new tool, as I see them, are: 1. Locate Python packages you want on the net. 2. Resolve dependencies between what you want, what you have, and what you need. 3. Download the appropriate packages to give you what you want. 4. Install those packages. Do we agree these are the primary features? I sense a consensus that the "install" part should be handled by distutils. Is that right? Other requirements we might lay down up front: 1. Should run on all platforms where Python runs. 2. Must support mirrors on the server side. 3. Need to include documentation along with source for packages. The features related to dependencies and downloads could be handled by a platform-specific packaging system, but integrating with all of the different options on all of the different platforms where Python runs increases the complexity of the new tool. (How do we handle RPMs on MacOS? How do we handle HQX files on Win32? What about for Pippy, where files as such might not even be appropriate? Does Jython have any special requirements?) Do we deal with this complexity by deciding on a per-platform basis what format to use, or do we force the user to specify the format they want downloaded? If we're modeling what we're doing based on CPAN, maybe we should look at that design. What format does CPAN use when downloading Perl packages? What features does CPAN have that we want? What does it not have that we want? Doug From jafo@tummy.com Mon Feb 26 23:50:09 2001 From: jafo@tummy.com (Sean Reifschneider) Date: Mon Feb 26 23:50:09 2001 Subject: [Distutils] Re: CPAN functionality for python - requirements In-Reply-To: <01022623140707.15046@branagan>; from doughellmann@home.com on Mon, Feb 26, 2001 at 10:45:38PM -0500 References: <01022606501904.15046@branagan> <20010226183606.F7531@tummy.com> <01022623140707.15046@branagan> Message-ID: <20010226214909.A10411@tummy.com> On Mon, Feb 26, 2001 at 10:45:38PM -0500, Doug Hellmann wrote: >The command "rpm -qa" only produces meaningful results on a system where RPM is >the default and primary package installation system. There are more OSes where >that is *not* the case (and where RPM isn't installed at all) than where it is. Apaprently I wasn't canonical enough... On an rpm-based system, it's bogus if "rpm -qa" doesn't list packages installed by CPAN. Just as it's bogus for the similar to happen on Debian, HP, Sun, Windows, etc... >It still isn't clear to me why we would want to use more than one distribution >package format. It appears that one argument is so that sysadmins can use >those tools to track what is installed on the system. But isn't that part of That's one argument, and I'd say it's a good one at that... Other reasons are: The admin doesn't have to learn new packaging formats to deal with the Python packages. If we don't leverage off the platform's package management tool, we have to write our own. Who's volunteering? If the packages aren't listed in the native packaging system, they are much more likely to get overlooked if admins search for things to update and the like. >1. Locate Python packages you want on the net. By "packages" do you mean modules only? I envision it also handling distribution of Python applications and even Python itself. >2. Resolve dependencies between what you want, what you have, and what you > need. >3. Download the appropriate packages to give you what you want. >4. Install those packages. > >Do we agree these are the primary features? Yes. >I sense a consensus that the "install" part should be handled by distutils. Is >that right? That seems to be where we diverge... If somone has made an RPM of it, I'd rather have that than some files winding up hanging around my file-system. >3. Need to include documentation along with source for packages. Not gonna happen... Until there's tools and standards for such documentation, it's not really possible to deal with them... In the cases where docstrings are used, installing the module produces the documentation, but that seems to be the exception... >complexity of the new tool. (How do we handle RPMs on MacOS? How do we handle My schema lists the package format, architecture and platform. If you search for a package for MacOS, and are returned an RPM, then apparently somone has built an RPM for MacOS. Your client should list the package formats that it desires and prefers, which means that if you don't have RPM on your Mac, you will pull down something else -- without penalizing those who *DO* have RPM running on their Mac... >HQX files on Win32? What about for Pippy, where files as such might not even >be appropriate? Does Jython have any special requirements?) Do we deal with My setup handles that -- set the platform to "pilot" and you get access to only the things that will work there (in theory). Again, there are two distinct things here. The archive network and the tools which make use of it... >this complexity by deciding on a per-platform basis what format to use, or do >we force the user to specify the format they want downloaded? You say "we force the user to specify". I say "we allow the user the choose". >If we're modeling what we're doing based on CPAN, maybe we should look at >that design. What format does CPAN use when downloading Perl packages? What >features does CPAN have that we want? What does it not have that we want? CPAN is a Unix-like directory structure, files are downloaded as .tar bundles which are extracted and a "perl Makefile.pl; make; make test; make install" is run. Does Pippy have "make"? Sean -- Gone Postal Sort: Iterate over elements, any element that is out of order you blow away. -- Evelyn, Kevin, and Sean, watching Monty Python and reading DDJ Sean Reifschneider, Inimitably Superfluous tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python From bsass@freenet.edmonton.ab.ca Tue Feb 27 02:56:01 2001 From: bsass@freenet.edmonton.ab.ca (Bruce Sass) Date: Tue Feb 27 02:56:01 2001 Subject: [Distutils] Re: CPAN functionality for python In-Reply-To: <20010226180935.B7531@tummy.com> Message-ID: On Mon, 26 Feb 2001, Sean Reifschneider wrote: > On Mon, Feb 26, 2001 at 11:14:35AM -0700, Bruce Sass wrote: > >from outside the archive. The translation from a generic pkg to a > >distro specific pkg would be a problem for the disto to solve > >(probably a matter of repacking info about the pkg, then using > > The only way having automaticly-generated packages be possible is > if we really force a conformance from the authors of the packages. > I've packaged quite a number of RPMs in my day, and you can get pretty > close to just wrapping a tar file with some meta-information in > some cases, but these are the minority. I have assumed that submitted packages would need to meet a standard of some sort - was that incorrect? > In most cases you have to provide patches which fix incorrect paths > in the tar file (for example, the Python tar file uses > "#!/usr/local/bin/python" all over the place). That is to be expected... and only the distro is really equipted to handle these issues. > If we can force the distributions to be more compatible with being > packaged, it may be very easy. Otherwise, we may have to rely on > people contributing their time to building RPMs if we want to have > RPMs. I'm not sure what you mean here. The distributions are the ones doing the packaging, the Python version of CPAN should supply all the bits needed for the distros to perform that function. > >only way I can see to do that is to not package to anyones spec, but > >provide enough information so that anyone can package to their > >fav spec. > > I don't agree... I believe that we should embrace the different packaging > formats, not act like they don't exist. Maybe we see "embracing" them a little differently. I believe providing everything they need to turn a Python module/package into their standard format, using their packaging and QC tools, does them more of a service than providing pre-built packages created by a third party. I'm not saying binary style packages shouldn't be supported, just that the emphasis should be on portable (in that they can easily be turned into a system's native pkg format) source packages. > If the distutils package can be > made to generate native RPMs, that's great. The *CLIENT* program should > dictate that. If it can handle turning a distutil package into an RPM, > it'll have it's preference be a distutil package. That sounds like exactly what I was enquiring about - as long as the distutils pkg provides enough info for every distro to create their own pkg. > I don't believe having the archive not able to deal with different package > formats is acceptable. If that were the case, then if the package didn't > provide enough information to build an RPM, the user would have no option > within the archive of working around it. Agreed. The submission should not be accepted if it does not contain enough information to be packaged. - Bruce From chris@voodooland.net Tue Feb 27 03:01:01 2001 From: chris@voodooland.net (Chris Watson) Date: Tue Feb 27 03:01:01 2001 Subject: [Distutils] Re: CPAN functionality for python In-Reply-To: Message-ID: > > I don't believe having the archive not able to deal with different package > > formats is acceptable. If that were the case, then if the package didn't > > provide enough information to build an RPM, the user would have no option > > within the archive of working around it. > > Agreed. The submission should not be accepted if it does not contain > enough information to be packaged. Not to toss a pebble in the still waters here. But it *might* be worth the effort to work with or at least get idea's from the openpackages people. Who are now wrestling with certain issues discussed here for handling a common package system across several platforms. Just a thought. Info can be found at http://www.openpackages.net. -- ============================================================================= -Chris Watson (316) 326-3862 | FreeBSD Consultant, FreeBSD Geek Work: scanner@jurai.net | Open Systems Inc., Wellington, Kansas Home: scanner@deceptively.shady.org | http://open-systems.net ============================================================================= WINDOWS: "Where do you want to go today?" LINUX: "Where do you want to go tomorrow?" BSD: "Are you guys coming or what?" ============================================================================= irc.openprojects.net #FreeBSD -Join the revolution! ICQ: 20016186 From bsass@freenet.edmonton.ab.ca Tue Feb 27 03:24:02 2001 From: bsass@freenet.edmonton.ab.ca (Bruce Sass) Date: Tue Feb 27 03:24:02 2001 Subject: [Distutils] Re: CPAN functionality for python In-Reply-To: <20010226182601.D7531@tummy.com> Message-ID: On Mon, 26 Feb 2001, Sean Reifschneider wrote: > On Mon, Feb 26, 2001 at 02:56:51AM -0700, Bruce Sass wrote: > >Do you mean: > > Ideally, it would be able to deal with both. The ability to download > a distutils package and produce a .deb or a .rpm is more a function of > the client program than of the archive network... It also largely depends > on distutils as well. So we are thinking along the same lines. > >As for the second... it seems like an awful waste of archive space to > >have the same code in many different formats, just because some want > >to put stuff in /usr/share and others in /usr/lib, etc. > > I'm really not familiar with .deb packages. Howeer, for RPMs, if the > author has included a .spec file in the tar file, you can simply run > "rpm -ta foo.tar.gz" and the results will (should) be a compiled binary > RPM. That is, if the author of the package has thought ahead about > producing an RPM. In a nutshell... Debian's equivalent of a .spec is the "debian" subdir in the source tree, containing administrivia and a "rules" makefile; doing "debian/rules" compiles, "debian/rules binary" compiles and packages, "debuild" compiles, packages and checks the package for errors and conformance to Policy. There can be a lot more to it, but nothing the author needs to worry about, in fact the author does not need to know anything about Debian. There are tools that create the debian dir and pull information from the source tree to customize the package, it is very easy to script so it can debianize anything thrown at it - provided the source tree is reasonably consistent. > The question becomes: Is it reasonable to require authors of packages > for the archive network to do the same? Convince me it is... No, it isn't. The authors should not need to know anything about the final format of their module, just that they need to provide specific pieces of information if they want to be included in the network (imo). Hmmm, so RPM based systems don't have a way to automatically generate a .spec file from a source tree... that would be a must. - Bruce From R.Liebscher@gmx.de Tue Feb 27 03:29:01 2001 From: R.Liebscher@gmx.de (Rene Liebscher) Date: Tue Feb 27 03:29:01 2001 Subject: [Distutils] Re: [Catalog-sig] [Announce] Distutils Catalog Prototype References: <3A9AE2C4.37CDFF22@digicool.com> Message-ID: <3A9B64EF.7E5B8D4A@gmx.de> Amos Latteier wrote: > > Hi Guys, > > I finally got off my butt and built a distutils catalog prototype. Now > for a limited time you can get to it at > > http://63.230.174.230:8080/ > > This is a Zope site that allows you to upload distutils archives and > search them. I added a basic HTML and XML interface. The XML interface > should be used by command-line clients. The user name and password for > uploading are both 'guest'. I am going to remove this account in a > couple days since it is fundamentally insecure. I mostly want to post > this now so folks can see what I'm doing. If you want to play with the > prototype more, get Zope 2.3 and install the > DistutilsArchive20010226.tgz file. > > I've preloaded the site with one archive (distutils 1.0.1). Try out the > site by uploading your own archives. If the site can't parse them take a > look in Parse.py and figure out what the problem is and send me a patch. > > Issues > > * Executing setup.py to find meta-data is insecure. Plus I've found > that many existing distutils archives don't actually expose the > meta-data very well this way. I propose that we fix this by having > distutils write a meta-data file when you build an archive. I tried to upload PyOpenGL and got the following result: Zope Error Zope has encountered an error while publishing this resource. Error Type: ImportError Error Value: No module named my_install_data .... Traceback (innermost last): File /home/amos/DistSite/lib/python/ZPublisher/Publish.py, line 223, in publish_module File /home/amos/DistSite/lib/python/ZPublisher/Publish.py, line 187, in publish File /home/amos/DistSite/lib/python/Zope/__init__.py, line 221, in zpublisher_exception_hook File /home/amos/DistSite/lib/python/ZPublisher/Publish.py, line 171, in publish File /home/amos/DistSite/lib/python/ZPublisher/mapply.py, line 160, in mapply (Object: addMethod) File /home/amos/DistSite/lib/python/ZPublisher/Publish.py, line 112, in call_object (Object: addMethod) File /home/amos/DistSite/lib/python/Products/DistutilsArchive/Archive.py, line 25, in addMethod File /home/amos/DistSite/lib/python/Products/DistutilsArchive/Archive.py, line 84, in update (Object: RoleManager) File /home/amos/DistSite/lib/python/Products/DistutilsArchive/Parse.py, line 69, in parse_meta_data File /usr/local/lib/python1.5/site-packages/distutils/core.py, line 209, in run_setup File /var/tmp/@11671.4/PyOpenGL-1.5.6/setup.py, line 16, in ? ImportError: (see above) ---------------------------------- PyOpenGL uses an external module to install its data. (It is imported but not used if you don't execute an install command.) If your system doesn't unpack all files before executing setup.py it will fail for all packages which import other python files in their setup.py. (And I think there are many such packages.) Kind regards Rene Liebscher From jafo@tummy.com Tue Feb 27 03:31:10 2001 From: jafo@tummy.com (Sean Reifschneider) Date: Tue Feb 27 03:31:10 2001 Subject: [Distutils] Re: CPAN functionality for python In-Reply-To: ; from bsass@freenet.edmonton.ab.ca on Tue, Feb 27, 2001 at 12:53:45AM -0700 References: <20010226180935.B7531@tummy.com> Message-ID: <20010227013019.T1781@tummy.com> (I'm going to remove the non-list entities in further messages -- if you want to follow this discussion you can find it on the lists. Not entirely sure how the CC list got so big in the first place) On Tue, Feb 27, 2001 at 12:53:45AM -0700, Bruce Sass wrote: >I have assumed that submitted packages would need to meet a >standard of some sort - was that incorrect? Obviously there will be some level of standards. I don't know that the tool should govern a policy decision though... It seems that you're pushing for a standard that's more than "just dump a .tar.gz here" and less than "schlep up tars and RPMs and .debs, and don't forget about HP-UX swinstall". Maybe something like those (defunct?) Linux Software Map entries? I don't know the answers to what the standards will be and how they'll be enforced... I do suspect that they'll change over time, so the tool should probably be flexible. >That is to be expected... and only the distro is really equipted to >handle these issues. I don't agree with that. Historicly, it's been fairly common for the package to say "Hey, I've got some paths I need in the various files. Here's the macros that represent them." >I'm not sure what you mean here. The distributions are the ones doing >the packaging, the Python version of CPAN should supply all the bits >needed for the distros to perform that function. What do you mean by "the distributions"? How will they do the packaging? Are you saying that RedHat should be creating the RPMs? That's just not how it's actually done in the real world... We just can't wait for the distribution vendors to decide to include them on their disc. It's not that sort of world. If tools can be provided so that a user can turn a .tar.gz file into a .RPM file easily, I'm all for it. What about systems where that isn't possible? As far as I can tell, my system doesn't *REQUIRE* the inclusion of RPMs, it just *ALLOWS* it. You seem to be arguing for the exclusion of that capability. Why? Sean -- The early bird may get the worm, but the second mouse gets the cheese. Sean Reifschneider, Inimitably Superfluous tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python From bsass@freenet.edmonton.ab.ca Tue Feb 27 03:37:01 2001 From: bsass@freenet.edmonton.ab.ca (Bruce Sass) Date: Tue Feb 27 03:37:01 2001 Subject: [Distutils] Re: CPAN functionality for python In-Reply-To: <20010226183138.E7531@tummy.com> Message-ID: On Mon, 26 Feb 2001, Sean Reifschneider wrote: > On Mon, Feb 26, 2001 at 11:57:36AM -0700, Bruce Sass wrote: > >Ya, RPMs are not known for doing a good job of dependencies. Debian > > They aren't? The example given was a situation where one could say that > RPM was doing *TOO* good a job of dependencies -- it was failing to > install because you didn't have what was needed. s/*TOO* good/overboard/ ;) > For quite a long time I was runing a 5.2 machine running the newest versions > of GTK, but older versions of glibc. Practicly none of the stock binary RPMs > would install on it because of that. > > However, if you pick up an SRPM, and do "rpm --rebuild package.src.rpm", it > will build the RPM against *EXACTLY* the set of libraries and files you > have. It works brilliantly. Agreed, so supply something that can be turned into an SRPM, or a Debian .dsc+.diff+.orig.tar.gz source package, or a MS whatever. I suppose one could provide SRPMs only - let everyone else do a conversion - but that is favoring a specific packaging system and could easily be misconstrued... which is why I suggested a neutral format. - Bruce From jafo@tummy.com Tue Feb 27 03:39:31 2001 From: jafo@tummy.com (Sean Reifschneider) Date: Tue Feb 27 03:39:31 2001 Subject: [Distutils] Re: CPAN functionality for python In-Reply-To: ; from bsass@freenet.edmonton.ab.ca on Tue, Feb 27, 2001 at 01:22:34AM -0700 References: <20010226182601.D7531@tummy.com> Message-ID: <20010227013832.U1781@tummy.com> On Tue, Feb 27, 2001 at 01:22:34AM -0700, Bruce Sass wrote: >Hmmm, so RPM based systems don't have a way to automatically generate >a .spec file from a source tree... that would be a must. RPMs include pieces of information such as: Description of package Summary of the package Source URL Copyright Type of package (network daemons, console tools, etc) What is needed to install it. What is needed to build it. Scripts used for adding and removing a package (probably less necessary for most modules packages, but is used for software for doing things like adding user IDs, inittab entries, and the like) What files are "documentation". Where to install files and what their permissions are. Which installed files are config files. Some of it is obviously superfluous, but much of it is not, and little of it can be automaticly pulled from an arbitrary tar file. Now, at the minimum, an RPM file consists of the files that make it up and the permissions, so you can fake a minimal RPM file if you need to. Sean -- Brooks's Law of Prototypes: Plan to throw one away, you will anyhow. Sean Reifschneider, Inimitably Superfluous tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python From jafo@tummy.com Tue Feb 27 03:42:10 2001 From: jafo@tummy.com (Sean Reifschneider) Date: Tue Feb 27 03:42:10 2001 Subject: [Distutils] Re: CPAN functionality for python In-Reply-To: ; from bsass@freenet.edmonton.ab.ca on Tue, Feb 27, 2001 at 01:35:56AM -0700 References: <20010226183138.E7531@tummy.com> Message-ID: <20010227014116.V1781@tummy.com> On Tue, Feb 27, 2001 at 01:35:56AM -0700, Bruce Sass wrote: >I suppose one could provide SRPMs only - let everyone else do a >conversion - but that is favoring a specific packaging system and >could easily be misconstrued... which is why I suggested a neutral >format. There is no such thing as a neutral format. If it uses .tar.gz files, then Slackware users will be winging that we swiped theirs. ;-) Sean -- It usually takes more than three weeks to prepare a good impromptu speech. -- Mark Twain Sean Reifschneider, Inimitably Superfluous tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python From R.Liebscher@gmx.de Tue Feb 27 04:05:02 2001 From: R.Liebscher@gmx.de (Rene Liebscher) Date: Tue Feb 27 04:05:02 2001 Subject: [Distutils] Re: CPAN functionality for python References: <14984.49127.853606.286789@w221.z064000254.bwi-md.dsl.cnc.net> <20010215170227.K23577@tummy.com> <20010226013903.B3221@tummy.com> Message-ID: <3A9B6D36.20B1B3C2@gmx.de> I don't know if I missed something, but if you want for all users install a package in their native format, why not so: 1. - download the binary in a format which can be unpacked by python/distutils on every system where this binary works (Linux -> tar.gz?) - download some meta data 2. extract the binaries in a directory 3. feed the meta data and the directory in distutils bdist_{rpm|debian|...) and create a package for the users prefered system (RPM,...) (You probably have to extend the bdist_* commands, but this way we also get people to write bdist_* commands for their platforms.) 4. use the finished package to install it Kind regards Rene Liebscher From jafo@tummy.com Tue Feb 27 04:29:11 2001 From: jafo@tummy.com (Sean Reifschneider) Date: Tue Feb 27 04:29:11 2001 Subject: [Distutils] Re: CPAN functionality for python In-Reply-To: <3A9B6D36.20B1B3C2@gmx.de>; from R.Liebscher@gmx.de on Tue, Feb 27, 2001 at 10:02:46AM +0100 References: <14984.49127.853606.286789@w221.z064000254.bwi-md.dsl.cnc.net> <20010215170227.K23577@tummy.com> <20010226013903.B3221@tummy.com> <3A9B6D36.20B1B3C2@gmx.de> Message-ID: <20010227022801.W1781@tummy.com> On Tue, Feb 27, 2001 at 10:02:46AM +0100, Rene Liebscher wrote: >3. feed the meta data and the directory in distutils > bdist_{rpm|debian|...) and create a package for the > users prefered system (RPM,...) As I understand it, the bdist_* can create a binary RPM from a set of pre-compiled binaries? Can this be used in conjunction with the other distutils tools to actually take the source file and turn it into the binaries on a given platform? If so, that's exactly what we need... Once I have the archive network back-end done, I'm going to start looking at the client. I'm getting fairly close to a bare-bones server as it is (without the admin tools -- adding a package requires some SQL). I've only looked passingly at the distutils, but will obviously have to get further into it shortly. Thanks, Sean -- Any sufficiently advanced technology is indistinguishable from a rigged demo. Sean Reifschneider, Inimitably Superfluous tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python From bsass@freenet.edmonton.ab.ca Tue Feb 27 04:35:00 2001 From: bsass@freenet.edmonton.ab.ca (Bruce Sass) Date: Tue Feb 27 04:35:00 2001 Subject: [Distutils] Re: CPAN functionality for python - requirements In-Reply-To: <01022623140707.15046@branagan> Message-ID: On Mon, 26 Feb 2001, Doug Hellmann wrote: > On Mon, 26 Feb 2001, Sean Reifschneider wrote: > > On Mon, Feb 26, 2001 at 06:45:33AM -0500, Doug Hellmann wrote: <...> > It still isn't clear to me why we would want to use more than one distribution > package format. It appears that one argument is so that sysadmins can use > those tools to track what is installed on the system. But isn't that part of > what this new thing (cyphon?) is supposed to do? Do we want Python to require > an external package management system on each platform where this tool is > supported, or do we want a Python-esq tool which just works the way it is > supposed to, in the same way, everywhere Python is available? I want users to be able to find and install Python stuff, preferably in a manner consistent with their native package management system. > But I think we're arguing over a design before we've worked out requirements. > The basic tasks of this new tool, as I see them, are: > > 1. Locate Python packages you want on the net. > 2. Resolve dependencies between what you want, what you have, and what you > need. There is no way an archive network can resolve dependencies because ther is no way for the archive network to know what is installed on a users machine (at least there had better not be!), or what packages are available to fulfil those dependencies. The best that can be done is to tell the target system what is required by the pkg being looked at. The dependencies get resolved when the native system decides on where it should fetch the other stuff from. Dependencies are a big problem. I've spoken with a local LUG member who was complaining that Debian does not know about the stuff he gets from CPAN, which makes mixing CPAN and debianized CPAN stuff a pain. Debian's solution is a user level program to debianize arbitrary CPAN modules... I want the same solution for Python modules, except I want it done transparently. > 3. Download the appropriate packages to give you what you want. > 4. Install those packages. Doesn't that mean the program needs to know about all the distros, and all the version of those distros, and all the supported ways a user can tweak the system. If you don't do that, and keep it up, then your installer is going to be breaking often (or always broken, depending on the user or distro). > Do we agree these are the primary features? Both 2. and 4. are the domain of the target system. 2. can be modified (or maybe just reworded). 4... > I sense a consensus that the "install" part should be handled by distutils. Is > that right? As long as distutils does not presume to know how to do the installation. > Other requirements we might lay down up front: > > 1. Should run on all platforms where Python runs. > 2. Must support mirrors on the server side. > 3. Need to include documentation along with source for packages. > > The features related to dependencies and downloads could be handled by a > platform-specific packaging system, but integrating with all of the different > options on all of the different platforms where Python runs increases the > complexity of the new tool. Downloading is independent of the native package management system. Complexity is only increased if the new tool thinks it should handle platform/OS/distro specific issues. > (How do we handle RPMs on MacOS? How do we handle > HQX files on Win32? What about for Pippy, where files as such might not even > be appropriate? Does Jython have any special requirements?) Do we deal with > this complexity by deciding on a per-platform basis what format to use, or do > we force the user to specify the format they want downloaded? You don't. You give the data to the target system and let it worry about the complexity of packaging. There is no reason for the archive network to know anything except where to send the data! > If we're modeling what we're doing based on CPAN, maybe we should look at > that design. What format does CPAN use when downloading Perl packages? What > features does CPAN have that we want? What does it not have that we want? Pros: don't know (long strings of punctuation and symbols give me a headache ;). Cons: Dependencies problems when mixed with modules fetched by other methods. - Bruce From R.Liebscher@gmx.de Tue Feb 27 04:46:01 2001 From: R.Liebscher@gmx.de (Rene Liebscher) Date: Tue Feb 27 04:46:01 2001 Subject: [Distutils] Re: CPAN functionality for python References: <14984.49127.853606.286789@w221.z064000254.bwi-md.dsl.cnc.net> <20010215170227.K23577@tummy.com> <20010226013903.B3221@tummy.com> <3A9B6D36.20B1B3C2@gmx.de> <20010227022801.W1781@tummy.com> Message-ID: <3A9B7713.334CD75D@gmx.de> Sean Reifschneider wrote: > > On Tue, Feb 27, 2001 at 10:02:46AM +0100, Rene Liebscher wrote: > >3. feed the meta data and the directory in distutils > > bdist_{rpm|debian|...) and create a package for the > > users prefered system (RPM,...) > > As I understand it, the bdist_* can create a binary RPM from a set of > pre-compiled binaries? Can this be used in conjunction with the other > distutils tools to actually take the source file and turn it into > the binaries on a given platform? The bdist_* commands do an install in a temporary directory and create from this a package (RPM,...). I think it is not a big deal to provide an already existing directory and create from it a package. ------------------------------------------------------- rob@cvs/distutils >python setup.py bdist_rpm --help Global options: --verbose (-v) run verbosely (default) --quiet (-q) run quietly (turns verbosity off) --dry-run (-n) don't actually do anything --help (-h) show detailed help message Options for 'bdist_rpm' command: --bdist-base base directory for creating built distributions --rpm-base base directory for creating RPMs (defaults to "rpm" under --bdist-base; must be specified for RPM 2) --dist-dir (-d) directory to put final RPM files in (and .spec files if --spec-only) --python path to Python interpreter to hard-code in the .spec file (default: "python") --fix-python hard-code the exact path to the current Python interpreter in the .spec file --spec-only only regenerate spec file --source-only only generate source RPM --binary-only only generate binary RPM --use-bzip2 use bzip2 instead of gzip to create source distribution --distribution-name name of the (Linux) distribution to which this RPM applies (*not* the name of the module distribution!) --group package classification [default: "Development/Libraries"] --release RPM release number --serial RPM serial number --vendor RPM "vendor" (eg. "Joe Blow ") [default: maintainer or author from setup script] --packager RPM packager (eg. "Jane Doe ")[default: vendor] --doc-files list of documentation files (space or comma-separated) --changelog path to RPM changelog --icon name of icon file --provides capabilities provided by this package --requires capabilities required by this package --conflicts capabilities which conflict with this package --build-requires capabilities required to build this package --obsoletes capabilities made obsolete by this package --keep-temp (-k) don't clean up RPM build directory --no-keep-temp clean up RPM build directory [default] --use-rpm-opt-flags compile with RPM_OPT_FLAGS when building from source RPM --no-rpm-opt-flags do not pass any RPM CFLAGS to compiler --rpm3-mode RPM 3 compatibility mode (default) --rpm2-mode RPM 2 compatibility mode usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...] or: setup.py --help [cmd1 cmd2 ...] or: setup.py --help-commands or: setup.py cmd --help ------------------------------------------------------------------------ May be it is enough to specify --bdist-base and --binary-only. The other option could get values from some downloaded meta data. > If so, that's exactly what we need... > > Once I have the archive network back-end done, I'm going to start looking at > the client. I'm getting fairly close to a bare-bones server as it is > (without the admin tools -- adding a package requires some SQL). > > I've only looked passingly at the distutils, but will obviously have to > get further into it shortly. > > Thanks, > Sean > -- > Any sufficiently advanced technology is indistinguishable from a rigged demo. > Sean Reifschneider, Inimitably Superfluous > tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python From bsass@freenet.edmonton.ab.ca Tue Feb 27 04:51:00 2001 From: bsass@freenet.edmonton.ab.ca (Bruce Sass) Date: Tue Feb 27 04:51:00 2001 Subject: [Distutils] Re: CPAN functionality for python - requirements In-Reply-To: <20010226214909.A10411@tummy.com> Message-ID: On Mon, 26 Feb 2001, Sean Reifschneider wrote: > On Mon, Feb 26, 2001 at 10:45:38PM -0500, Doug Hellmann wrote: <...> > >I sense a consensus that the "install" part should be handled by distutils. Is > >that right? > > That seems to be where we diverge... If somone has made an RPM of it, I'd > rather have that than some files winding up hanging around my file-system. Even if the RPM was made by someone who didn't have a clue? Wouldn't it be better to have a standard way to convert a Python package into an RPM, that way any packaging problems will be seen by everyone and the fix will be fast. > >3. Need to include documentation along with source for packages. > > Not gonna happen... Until there's tools and standards for such documentation, > it's not really possible to deal with them... In the cases where docstrings > are used, installing the module produces the documentation, but that seems > to be the exception... Huh, there are tools and standards. Isn't /usr/shar/doc/package the FHS place for general package specific docs, and before that it was /usr/doc/package (FSSTND), MS users just put them into the installation dir. Special formats usually have a place to reside, but I wouldn't expect an archive network or its tools to know or care what and where they are. > CPAN is a Unix-like directory structure, files are downloaded as .tar bundles > which are extracted and a "perl Makefile.pl; make; make test; make install" > is run. Does Pippy have "make"? The above is what Debian appears to have replaced - *ifiact*, because it was not versatile enough, it thinks it knows how things should be done and is wrong. - Bruce From bsass@freenet.edmonton.ab.ca Tue Feb 27 04:56:00 2001 From: bsass@freenet.edmonton.ab.ca (Bruce Sass) Date: Tue Feb 27 04:56:00 2001 Subject: [Distutils] Re: CPAN functionality for python In-Reply-To: <20010226183857.G7531@tummy.com> Message-ID: This confirms what I got out of another list member in p-email... all the problems surrounding RPMs I hear about are most likely the result of inexperienced packagers. Debian can have the same problems, but the packaging tools are thorough enough that things usually work, even if the result is really ugly (e.g., the Active State py-2.0 .debs ). -- On Mon, 26 Feb 2001, Sean Reifschneider wrote: > On Mon, Feb 26, 2001 at 12:22:47PM -0700, Bruce Sass wrote: > >I think the problem with RPMs is that they depend on a specific > >package, instead of depending on the contents of a package. > > In RPM, you can let the packaging system decide what revisions of > various packages are needed (by looking at the results, it will > find out what libraries it links against, what's in the she-bang > (#!) lines, etc). That mechanism is usually the most paranoid > about versions (if we linked against glibc 2.1.7.9.pi, then that's > exactly what we expect). > > On the other hand, if you know better, as an application developer, > you can include something like: > > Requires: python >= 1.5.2 > > in the .spec file which says "I know that 1.5.1 won't work, but I hope > that 1.6 won't break this package". > > Sean > From doughellmann@home.com Tue Feb 27 06:49:11 2001 From: doughellmann@home.com (Doug Hellmann) Date: Tue Feb 27 06:49:11 2001 Subject: [Distutils] Re: CPAN functionality for python - requirements In-Reply-To: <20010226183606.F7531@tummy.com> References: <01022606501904.15046@branagan> <20010226183606.F7531@tummy.com> Message-ID: <01022623140707.15046@branagan> On Mon, 26 Feb 2001, Sean Reifschneider wrote: > On Mon, Feb 26, 2001 at 06:45:33AM -0500, Doug Hellmann wrote: > >What if Python kept up with its own packages and modules like XEmacs does? We > >could then use the same format on all platforms -- it might even be something > >we roll ourself to make sure we can unpack it with *only* Python installed on a > >platform. > > My understanding is that that's what distutils is meant to do, to an extent. > The problem is that it's not entirely unreasonable to expect that an > "rpm -qa" produce a list of all the software that's installed on your > box. If you don't use an RPM, you can't really make use of that, and > packages WILL get dropped after a re-install. The command "rpm -qa" only produces meaningful results on a system where RPM is the default and primary package installation system. There are more OSes where that is *not* the case (and where RPM isn't installed at all) than where it is. It still isn't clear to me why we would want to use more than one distribution package format. It appears that one argument is so that sysadmins can use those tools to track what is installed on the system. But isn't that part of what this new thing (cyphon?) is supposed to do? Do we want Python to require an external package management system on each platform where this tool is supported, or do we want a Python-esq tool which just works the way it is supposed to, in the same way, everywhere Python is available? But I think we're arguing over a design before we've worked out requirements. The basic tasks of this new tool, as I see them, are: 1. Locate Python packages you want on the net. 2. Resolve dependencies between what you want, what you have, and what you need. 3. Download the appropriate packages to give you what you want. 4. Install those packages. Do we agree these are the primary features? I sense a consensus that the "install" part should be handled by distutils. Is that right? Other requirements we might lay down up front: 1. Should run on all platforms where Python runs. 2. Must support mirrors on the server side. 3. Need to include documentation along with source for packages. The features related to dependencies and downloads could be handled by a platform-specific packaging system, but integrating with all of the different options on all of the different platforms where Python runs increases the complexity of the new tool. (How do we handle RPMs on MacOS? How do we handle HQX files on Win32? What about for Pippy, where files as such might not even be appropriate? Does Jython have any special requirements?) Do we deal with this complexity by deciding on a per-platform basis what format to use, or do we force the user to specify the format they want downloaded? If we're modeling what we're doing based on CPAN, maybe we should look at that design. What format does CPAN use when downloading Perl packages? What features does CPAN have that we want? What does it not have that we want? Doug From doughellmann@home.com Tue Feb 27 06:49:24 2001 From: doughellmann@home.com (Doug Hellmann) Date: Tue Feb 27 06:49:24 2001 Subject: [Distutils] Re: CPAN functionality for python - requirements In-Reply-To: References: Message-ID: <01022706484400.20069@branagan> On Tue, 27 Feb 2001, Bruce Sass wrote: > On Mon, 26 Feb 2001, Doug Hellmann wrote: > > > On Mon, 26 Feb 2001, Sean Reifschneider wrote: > > > On Mon, Feb 26, 2001 at 06:45:33AM -0500, Doug Hellmann wrote: > <...> > > It still isn't clear to me why we would want to use more than one distribution > > package format. It appears that one argument is so that sysadmins can use > > those tools to track what is installed on the system. But isn't that part of > > what this new thing (cyphon?) is supposed to do? Do we want Python to require > > an external package management system on each platform where this tool is > > supported, or do we want a Python-esq tool which just works the way it is > > supposed to, in the same way, everywhere Python is available? > > I want users to be able to find and install Python stuff, preferably > in a manner consistent with their native package management system. > > > But I think we're arguing over a design before we've worked out requirements. > > The basic tasks of this new tool, as I see them, are: > > > > 1. Locate Python packages you want on the net. > > 2. Resolve dependencies between what you want, what you have, and what you > > need. > > There is no way an archive network can resolve dependencies because No, the network doesn't. But the network by itself isn't going to do most of these things. The client app would need to resolve the dependencies. If I tell the client to find and download PmwContribD, it needs to figure out that I also need Pmw (and perhaps a minimum version) and either suggest that I go ahead and install it or fail with a message telling me to install that first depending on the command line arguments given. > ther is no way for the archive network to know what is installed on a > users machine (at least there had better not be!), or what packages > are available to fulfil those dependencies. The best that can > be done is to tell the target system what is required by the pkg being > looked at. The dependencies get resolved when the native system > decides on where it should fetch the other stuff from. It sounds like the knows how to find a particular package, but getting it installed is your problem. We can make it smarter than that. > Dependencies are a big problem. I've spoken with a local LUG member > who was complaining that Debian does not know about the stuff he gets from > CPAN, which makes mixing CPAN and debianized CPAN stuff a pain. > Debian's solution is a user level program to debianize arbitrary CPAN > modules... I want the same solution for Python modules, except I want > it done transparently. I can see that as somewhat desirable, and it could be implemented, but do you make the cyphon client download the right package format or convert a standard distro into the right format? Doesn't the user have to specify a package type? > > 3. Download the appropriate packages to give you what you want. > > 4. Install those packages. > > Doesn't that mean the program needs to know about all the distros, and > all the version of those distros, and all the supported ways a user > can tweak the system. If you don't do that, and keep it up, then your > installer is going to be breaking often (or always broken, depending > on the user or distro). > > > Do we agree these are the primary features? > > Both 2. and 4. are the domain of the target system. 2. can be > modified (or maybe just reworded). 4... What I want is a single app I can use that will do all of those things. I want to type one command and have it find and install software on my system that will allow me to use some Python package that I don't have yet. That app would have a client side and server side, but as a user I don't care about the server side unless I want to specify a specific mirror to use by default. > > I sense a consensus that the "install" part should be handled by distutils. Is > > that right? > > As long as distutils does not presume to know how to do the > installation. How can it do the install if it doesn't know how? It sounds like you do *not* want distutils to do the install, and that you cyphon to download/create a platform-specific package to be installed. Since we *have* distutils as an ostensibly standard installation system for Python modules, shouldn't we use it? > > Other requirements we might lay down up front: > > > > 1. Should run on all platforms where Python runs. > > 2. Must support mirrors on the server side. > > 3. Need to include documentation along with source for packages. > > > > The features related to dependencies and downloads could be handled by a > > platform-specific packaging system, but integrating with all of the different > > options on all of the different platforms where Python runs increases the > > complexity of the new tool. > > Downloading is independent of the native package management system. No, it isn't. Even if *all* the tool does is download things I told it to find and write them to the current directory, it has to know what format I like or it may download (for example) debian packages for my Mac -- and then I won't get useful results from the tool. > Complexity is only increased if the new tool thinks it should handle > platform/OS/distro specific issues. It *must* handle those issues to be useful. > > (How do we handle RPMs on MacOS? How do we handle > > HQX files on Win32? What about for Pippy, where files as such might not even > > be appropriate? Does Jython have any special requirements?) Do we deal with > > this complexity by deciding on a per-platform basis what format to use, or do > > we force the user to specify the format they want downloaded? > > You don't. You give the data to the target system and let it worry > about the complexity of packaging. What is the "target system," the OS? > There is no reason for the archive network to know anything except > where to send the data! Right, but we do need a client so that users have a way to interact with the network. Wait -- maybe that's where the disconnect is! I'm describing features of the system from the client-side view, assuming the network would provide services to support those features. It sounds like you are describing features of the network. We should determine the requirements from the user perspective first, and design something to meet those requirements. Designs for the network services will follow from the requirements. > > If we're modeling what we're doing based on CPAN, maybe we should look at > > that design. What format does CPAN use when downloading Perl packages? What > > features does CPAN have that we want? What does it not have that we want? > > Pros: > don't know (long strings of punctuation and symbols give me a headache ;). Heh, tell me about it. I can't remember now whether CPAN did installations or just downloaded stuff and made you figure out how to install it. > Cons: > Dependencies problems when mixed with modules fetched by other methods. Right, so we need a single way of determining, maintaining, and resolving dependencies. Doug From doughellmann@home.com Tue Feb 27 06:57:11 2001 From: doughellmann@home.com (Doug Hellmann) Date: Tue Feb 27 06:57:11 2001 Subject: [Distutils] Re: CPAN functionality for python - requirements In-Reply-To: <20010226214909.A10411@tummy.com> References: <01022623140707.15046@branagan> <20010226214909.A10411@tummy.com> Message-ID: <01022706564001.20069@branagan> On Mon, 26 Feb 2001, Sean Reifschneider wrote: > On Mon, Feb 26, 2001 at 10:45:38PM -0500, Doug Hellmann wrote: > >The command "rpm -qa" only produces meaningful results on a system where RPM is > >the default and primary package installation system. There are more OSes where > >that is *not* the case (and where RPM isn't installed at all) than where it is. > > Apaprently I wasn't canonical enough... On an rpm-based system, it's > bogus if "rpm -qa" doesn't list packages installed by CPAN. Just as it's > bogus for the similar to happen on Debian, HP, Sun, Windows, etc... > > >It still isn't clear to me why we would want to use more than one distribution > >package format. It appears that one argument is so that sysadmins can use > >those tools to track what is installed on the system. But isn't that part of > > That's one argument, and I'd say it's a good one at that... Other reasons > are: > > The admin doesn't have to learn new packaging formats to deal with the > Python packages. > > If we don't leverage off the platform's package management tool, we have > to write our own. Who's volunteering? > > If the packages aren't listed in the native packaging system, they are > much more likely to get overlooked if admins search for things to > update and the like. > > >1. Locate Python packages you want on the net. > > By "packages" do you mean modules only? I envision it also handling > distribution of Python applications and even Python itself. Sure, all of that. Anything which uses distutils for installation, right? > >2. Resolve dependencies between what you want, what you have, and what you > > need. > >3. Download the appropriate packages to give you what you want. > >4. Install those packages. > > > >Do we agree these are the primary features? > > Yes. > > >I sense a consensus that the "install" part should be handled by distutils. Is > >that right? > > That seems to be where we diverge... If somone has made an RPM of it, I'd > rather have that than some files winding up hanging around my file-system. So why did distutils make it into the Python core? > >3. Need to include documentation along with source for packages. > > Not gonna happen... Until there's tools and standards for such documentation, > it's not really possible to deal with them... In the cases where docstrings > are used, installing the module produces the documentation, but that seems > to be the exception... Ok, so make that: 3. Need to support including documentation along with source for packages when documentation is available. > >complexity of the new tool. (How do we handle RPMs on MacOS? How do we handle > > My schema lists the package format, architecture and platform. If you > search for a package for MacOS, and are returned an RPM, then apparently > somone has built an RPM for MacOS. Your client should list the package > formats that it desires and prefers, which means that if you don't have > RPM on your Mac, you will pull down something else -- without penalizing > those who *DO* have RPM running on their Mac... But that leaves it up to the person posting the package to make distributions in formats for any platforms where folks might want to install their stuff, or to have someone *else* create those packages. That may lead to a situation where someone believes that they cannot install my package on MacOS because I didn't make an HQX file (or whatever) when I posted HappyDoc. > >HQX files on Win32? What about for Pippy, where files as such might not even > >be appropriate? Does Jython have any special requirements?) Do we deal with > > My setup handles that -- set the platform to "pilot" and you get access to > only the things that will work there (in theory). > > Again, there are two distinct things here. The archive network and the > tools which make use of it... Right. We need to make it easy for the user to do the right thing. > >this complexity by deciding on a per-platform basis what format to use, or do > >we force the user to specify the format they want downloaded? > > You say "we force the user to specify". I say "we allow the user the choose". I can go with "allow" but if we do not help then the user must specify in order to get useful results from the tool, and that becomes "force." > >If we're modeling what we're doing based on CPAN, maybe we should look at > >that design. What format does CPAN use when downloading Perl packages? What > >features does CPAN have that we want? What does it not have that we want? > > CPAN is a Unix-like directory structure, files are downloaded as .tar bundles > which are extracted and a "perl Makefile.pl; make; make test; make install" So, it sounds like their tool just does the download and then they use the equivalent of distutils to do the installation. Is that all we want? > is run. Does Pippy have "make"? I don't know about that. Maybe it isn't even appropriate to support Pippy, I'm not sure. Doug From guido@digicool.com Tue Feb 27 09:05:01 2001 From: guido@digicool.com (Guido van Rossum) Date: Tue Feb 27 09:05:01 2001 Subject: [Distutils] Re: CPAN functionality for python In-Reply-To: Your message of "Tue, 27 Feb 2001 02:54:29 MST." References: Message-ID: <200102270210.VAA01457@cj20424-a.reston1.va.home.com> > This confirms what I got out of another list member in p-email... > all the problems surrounding RPMs I hear about are most likely the > result of inexperienced packagers. That's easy to say. If lots of people contributed Python modules that were coded in a non-portable way, would we blame those users? No! We'd look for a way to encourage newbies to do the right thing! I still blame RPM for making it too damn hard to do the right thing. RPM mist be the Perl of packaging. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From mwa@gate.net Tue Feb 27 09:17:10 2001 From: mwa@gate.net (Mark W. Alexander) Date: Tue Feb 27 09:17:10 2001 Subject: [Distutils] Re: CPAN functionality for python - requirements In-Reply-To: <01022706484400.20069@branagan> Message-ID: On Tue, 27 Feb 2001, Doug Hellmann wrote: > Subject: [Distutils] Re: CPAN functionality for python - requirements > > I can't remember now whether CPAN did installations or just downloaded stuff > and made you figure out how to install it. (My understanding of) CPAN is that it's a repository of things that install a standard way. You download what you want and perl Makefile.pl make install Mark From bsass@freenet.edmonton.ab.ca Tue Feb 27 09:38:01 2001 From: bsass@freenet.edmonton.ab.ca (Bruce Sass) Date: Tue Feb 27 09:38:01 2001 Subject: [Distutils] Re: CPAN functionality for python - requirements In-Reply-To: <01022706484400.20069@branagan> Message-ID: On Tue, 27 Feb 2001, Doug Hellmann wrote: > On Tue, 27 Feb 2001, Bruce Sass wrote: > > On Mon, 26 Feb 2001, Doug Hellmann wrote: > > > On Mon, 26 Feb 2001, Sean Reifschneider wrote: > > > > On Mon, Feb 26, 2001 at 06:45:33AM -0500, Doug Hellmann wrote: > > <...> > > There is no way an archive network can resolve dependencies because > > No, the network doesn't. But the network by itself isn't going to do most of > these things. The client app would need to resolve the dependencies. > > If I tell the client to find and download PmwContribD, it needs to figure out > that I also need Pmw (and perhaps a minimum version) and either suggest that I > go ahead and install it or fail with a message telling me to install that first > depending on the command line arguments given. You are easily pleased. :) > > ther is no way for the archive network to know what is installed on a > > users machine (at least there had better not be!), or what packages > > are available to fulfil those dependencies. The best that can > > be done is to tell the target system what is required by the pkg being > > looked at. The dependencies get resolved when the native system > > decides on where it should fetch the other stuff from. > > It sounds like the knows how to find a particular package, but getting it > installed is your problem. We can make it smarter than that. Not "your", the native package management system's (and they have been doing it a relatively long time). > > Dependencies are a big problem. I've spoken with a local LUG member > > who was complaining that Debian does not know about the stuff he gets from > > CPAN, which makes mixing CPAN and debianized CPAN stuff a pain. > > Debian's solution is a user level program to debianize arbitrary CPAN > > modules... I want the same solution for Python modules, except I want > > it done transparently. > > I can see that as somewhat desirable, and it could be implemented, but do > you make the cyphon client download the right package format or convert > a standard distro into the right format? Doesn't the user have to specify a > package type? I prefer building from a generic package to a system specific package (QC is reduced to checking the contents of the generic source package and having a good package making script, hopefully provided by the same group of people who control the format of the package), but there is no reason for not supporting both (except maybe room to store it all), and in some cases it would be desirable. Sure the user has to specify a package type, _if_ they want a specific type of package, that info can be included in the name of the file being requested (built up from the name of the python piece, a client id, etc.) > > > 3. Download the appropriate packages to give you what you want. > > > 4. Install those packages. > > > > Doesn't that mean the program needs to know about all the distros, and > > all the version of those distros, and all the supported ways a user > > can tweak the system. If you don't do that, and keep it up, then your > > installer is going to be breaking often (or always broken, depending > > on the user or distro). > > > > > Do we agree these are the primary features? > > > > Both 2. and 4. are the domain of the target system. 2. can be > > modified (or maybe just reworded). 4... > > What I want is a single app I can use that will do all of those things. I want > to type one command and have it find and install software on my system that > will allow me to use some Python package that I don't have yet. That app would > have a client side and server side, but as a user I don't care about the server > side unless I want to specify a specific mirror to use by default. > > > > I sense a consensus that the "install" part should be handled by distutils. Is > > > that right? > > > > As long as distutils does not presume to know how to do the > > installation. > > How can it do the install if it doesn't know how? > > It sounds like you do *not* want distutils to do the install, and that you > cyphon to download/create a platform-specific package to be installed. Since we > *have* distutils as an ostensibly standard installation system for Python > modules, shouldn't we use it? I would not use it if it means bypassing the native package management system, and would be willing to wait for "official" packages rather than use third-party packages. Best of all scenarios would be if all the Python programmers used the same system I did and spit out packages in my favorite format, next best is having a package built by someone who knows my system just as well or better than I, then their proxy in the form of a scripted build, followed by someone I don't know building the package, a tarball or some other package format I can handle. The first is a dream, the second could happen now and then if I was lucky, so I'll try for my third choice (and it sounds like distutils can handle it). > > > Other requirements we might lay down up front: > > > > > > 1. Should run on all platforms where Python runs. > > > 2. Must support mirrors on the server side. > > > 3. Need to include documentation along with source for packages. > > > > > > The features related to dependencies and downloads could be handled by a > > > platform-specific packaging system, but integrating with all of the different > > > options on all of the different platforms where Python runs increases the > > > complexity of the new tool. > > > > Downloading is independent of the native package management system. > > No, it isn't. Even if *all* the tool does is download things I told it to find > and write them to the current directory, it has to know what format I like or > it may download (for example) debian packages for my Mac -- and then I won't > get useful results from the tool. Build up the name of the download from the name of the package and system specific information, just like is done with the names of .deb and .rpm packages. > > Complexity is only increased if the new tool thinks it should handle > > platform/OS/distro specific issues. > > It *must* handle those issues to be useful. Isn't that is a lot of duplicated work. > > > (How do we handle RPMs on MacOS? How do we handle > > > HQX files on Win32? What about for Pippy, where files as such might not even > > > be appropriate? Does Jython have any special requirements?) Do we deal with > > > this complexity by deciding on a per-platform basis what format to use, or do > > > we force the user to specify the format they want downloaded? > > > > You don't. You give the data to the target system and let it worry > > about the complexity of packaging. > > What is the "target system," the OS? The system the python package is being downloaded to. > > There is no reason for the archive network to know anything except > > where to send the data! > > Right, but we do need a client so that users have a way to interact with the > network. > > Wait -- maybe that's where the disconnect is! I'm describing features of the > system from the client-side view, assuming the network would provide services > to support those features. It sounds like you are describing features of the > network. We should determine the requirements from the user perspective first, > and design something to meet those requirements. Designs for the network > services will follow from the requirements. I see them as two pieces of the same puzzle. The third, and most important, is the data structure used to represent the python package. > > > If we're modeling what we're doing based on CPAN, maybe we should look at > > > that design. What format does CPAN use when downloading Perl packages? What > > > features does CPAN have that we want? What does it not have that we want? > > > > Pros: > > don't know (long strings of punctuation and symbols give me a headache ;). > > Heh, tell me about it. > > I can't remember now whether CPAN did installations or just downloaded stuff > and made you figure out how to install it. > > > Cons: > > Dependencies problems when mixed with modules fetched by other methods. > > Right, so we need a single way of determining, maintaining, and > resolving dependencies. Or a way of enabling the native tools so they can do it. I would not want to accept the job of keeping up with changing packaging standards for different systems, and don't expect that anyone else would want to (or be able to keep it up). - Bruce From doughellmann@home.com Tue Feb 27 10:33:10 2001 From: doughellmann@home.com (Doug Hellmann) Date: Tue Feb 27 10:33:10 2001 Subject: [Distutils] Re: CPAN functionality for python - requirements In-Reply-To: References: Message-ID: <01022710324002.20069@branagan> On Tue, 27 Feb 2001, Mark W. Alexander wrote: > I have no idea how this cc: list came to be. I'm adding catalog-sig > and will drop individuals on subsequent posts.... I've done that, too. It was getting a bit long. > On Tue, 27 Feb 2001, Bruce Sass wrote: > > Subject: [Distutils] Re: CPAN functionality for python - requirements > > > > > I sense a consensus that the "install" part should be handled by distutils. Is > > > that right? > > > > As long as distutils does not presume to know how to do the > > installation. > > > > > Other requirements we might lay down up front: > > > > > > 1. Should run on all platforms where Python runs. > > > 2. Must support mirrors on the server side. > > > 3. Need to include documentation along with source for packages. > > > > > > The features related to dependencies and downloads could be handled by a > > > platform-specific packaging system, but integrating with all of the different > > > options on all of the different platforms where Python runs increases the > > > complexity of the new tool. > > It seems there's a misunderstanding of what distutils does. It provides > a standard mechanism for producing installable python packages. IOW, I think the misunderstanding was on my part, and between your description and reviewing the distutils help I think I'm straightened out now. With that said, I agree that the network needs to allow, optionally, for binary distributions for platforms where building binary distributions is unusually difficult (like having no compiler on Win32). All packages should be available in source form (assuming some OS license) by default. The client should offer the user the option of downloading any available package format, which can then be installed using a separate tool ("./setup.py install" or rpm or whatever). If as a user, I choose to download an RPM, that's up to me. If no RPM is available, the thing I do download should be able to *make* an RPM, so that's where distutils comes in. So, we don't quite get to a *standard* set of download and install instructions because downloading different package types requires different installation steps. By trading that off we get package management using the user's favorite package system. That seems to be a reasonable trade. > So the catalog client will need to be able to acquire dependency > information from the network archive in order to ascertain whether > the module is buildable on the target system. If it is, it will > also need to acquire, invoke distutils to build and package, and > invoke the package manager to install all dependencies in order. I'm going to back off my request that this be an all-the-way tool. Let's stick to finding a package and all of its dependencies, and leave the installation to the packages which are downloaded. As a Python user, I may not have system privileges sufficient to allow me to install a downloaded package. As a system administrator, I may not know how to use this tool to find packages. So, the Python user can find the package and provide it to the sysadmin for installation. Nice and clean. Optionally, there could be an argument which would invoke an installation command for the package(s) being downloaded. > If it is not buildable, then it would need to seek out pre-built > binaries, download and install them. So, either the network archive > interfaces with distutils such that a back-end can produce binaries, Can I build binaries for all platforms on whatever platform I use for the network archive? > OR archive maintainers manually produce binaries for whatever > platforms they choose to support, OR automatically. If the source distribution is always available, part of the mirroring process could be to convert the source distributions into binary distributions for the platform serviced by the mirror site. > OR people without development tools (Windows, Macs(?), disk-space tight > OS's) are SOL. OR developers who do not have access to platforms where binary distributions are prefered enlist other interested parties to assist them in preparing binary packages. OR third party volunteers post binary distributions to other sites on the network > I hope this clarifies distutils (potential) role a bit. Significantly. Thanks, Doug From mwa@gate.net Tue Feb 27 10:55:10 2001 From: mwa@gate.net (Mark W. Alexander) Date: Tue Feb 27 10:55:10 2001 Subject: [Catalog-sig] Re: [Distutils] Re: CPAN functionality for python - requirements In-Reply-To: <01022710324002.20069@branagan> Message-ID: On Tue, 27 Feb 2001, Doug Hellmann wrote: > Date: Tue, 27 Feb 2001 10:17:06 -0500 > From: Doug Hellmann > To: python-list@python.org, distutils-sig@python.org, catalog-sig@python.org > Subject: [Catalog-sig] Re: [Distutils] Re: CPAN functionality for python > - requirements > [snip] > > It seems there's a misunderstanding of what distutils does. It provides > > a standard mechanism for producing installable python packages. IOW, > > I think the misunderstanding was on my part, and between your description and > reviewing the distutils help I think I'm straightened out now. > > With that said, I agree that the network needs to allow, optionally, for binary > distributions for platforms where building binary distributions is unusually > difficult (like having no compiler on Win32). All packages should be available > in source form (assuming some OS license) by default. The client should offer > the user the option of downloading any available package format, which can then > be installed using a separate tool ("./setup.py install" or rpm or whatever). > > If as a user, I choose to download an RPM, that's up to me. If no RPM is > available, the thing I do download should be able to *make* an RPM, so that's > where distutils comes in. Agreed. [snip] > I'm going to back off my request that this be an all-the-way tool. Let's stick > to finding a package and all of its dependencies, and leave the installation to > the packages which are downloaded. As a Python user, I may not have system > privileges sufficient to allow me to install a downloaded package. As a system > administrator, I may not know how to use this tool to find packages. So, the > Python user can find the package and provide it to the sysadmin for > installation. Nice and clean. > > Optionally, there could be an argument which would invoke an installation > command for the package(s) being downloaded. Yes, after I sent this off, I started thinking this as well. It's better as well for sites with multiple machines. Go get it once and install everywhere. > > If it is not buildable, then it would need to seek out pre-built > > binaries, download and install them. So, either the network archive > > interfaces with distutils such that a back-end can produce binaries, > > Can I build binaries for all platforms on whatever platform I use for the > network archive? That's an interesting question. What would it take to include cross-compiling support in distutils? > > OR archive maintainers manually produce binaries for whatever > > platforms they choose to support, > > OR automatically. If the source distribution is always available, part of the > mirroring process could be to convert the source distributions into binary > distributions for the platform serviced by the mirror site. That was my original thought. I would hate to see, however, that some platform gets "slighted" because there's no one to run a mirror for them. > > OR people without development tools (Windows, Macs(?), disk-space tight > > OS's) are SOL. > > OR developers who do not have access to platforms where binary distributions > are prefered enlist other interested parties to assist them in preparing binary > packages. > > OR third party volunteers post binary distributions to other sites on the > network These last 2 are where I hoped the combination of distutils and a network archive would reduce the number of volunteers needed. As long as distutils can produce a binary for a particular platform/ package manager combination, there's really no need for a dedicated (official or UNofficial) "package mantainer". All that's needed is a place where "python setup.py --bdist format=MyPackager" works. How about an interarchive-API (or redirector?), so archives can forward requests for binary packages to other sites known to support that type of binary package? Mark From mwa@gate.net Tue Feb 27 11:31:34 2001 From: mwa@gate.net (Mark W. Alexander) Date: Tue Feb 27 11:31:34 2001 Subject: [Distutils] Re: CPAN functionality for python - requirements In-Reply-To: Message-ID: I have no idea how this cc: list came to be. I'm adding catalog-sig and will drop individuals on subsequent posts.... On Tue, 27 Feb 2001, Bruce Sass wrote: > Subject: [Distutils] Re: CPAN functionality for python - requirements > > > I sense a consensus that the "install" part should be handled by distutils. Is > > that right? > > As long as distutils does not presume to know how to do the > installation. > > > Other requirements we might lay down up front: > > > > 1. Should run on all platforms where Python runs. > > 2. Must support mirrors on the server side. > > 3. Need to include documentation along with source for packages. > > > > The features related to dependencies and downloads could be handled by a > > platform-specific packaging system, but integrating with all of the different > > options on all of the different platforms where Python runs increases the > > complexity of the new tool. It seems there's a misunderstanding of what distutils does. It provides a standard mechanism for producing installable python packages. IOW, exactly what's under discussion here. It does so by the package author providing code and meta-data. Distutils uses the code to produce the python/C modules, and the meta-data to produce binary packages in _any_supported_format. Currently, RPM, tar.gz, and Windows installers are included. Solaris and HP-UX package support is available in horrendously hacked format, but it's been demonstrated to be possible. I imagine dpkg to be of minimal difficulty for someone who understands debian packaging. Note that, theoretically at least, the author needs no knowledge of packaging for distutils to do it's work. All they need to know is what meta-data is required by distutils. (I think there are still some issues regarding a "standard" meta-data base that is necessary to support "all" packagers, but that's not in insurmountable problem.) So distutils can be used to do the installation. The problem, however, is in the capabilities of the target system. Is there a compiler available for compiled modules? If not, the source is worthless, and the user is forced to rely on binary packages (a significant number of Windows users, I'd imagine), driving a requirement that the network provide (at least some) binary packages in directly downloadable form. OTOH, if compile time requirements are met (compiler, linker, SWIG, whatever else the author uses), then distutils can easily produce a binary package on the target machine, as long as the target's native package manager is supported by distutils. And as long as all dependency requirements are met. So the catalog client will need to be able to acquire dependency information from the network archive in order to ascertain whether the module is buildable on the target system. If it is, it will also need to acquire, invoke distutils to build and package, and invoke the package manager to install all dependencies in order. If it is not buildable, then it would need to seek out pre-built binaries, download and install them. So, either the network archive interfaces with distutils such that a back-end can produce binaries, OR archive maintainers manually produce binaries for whatever platforms they choose to support, OR people without development tools (Windows, Macs(?), disk-space tight OS's) are SOL. I hope this clarifies distutils (potential) role a bit. mwa From efm@tummy.com Tue Feb 27 11:32:03 2001 From: efm@tummy.com (Evelyn Mitchell) Date: Tue Feb 27 11:32:03 2001 Subject: [Distutils] Re: CPAN functionality for python - requirements In-Reply-To: ; from mwa@gate.net on Tue, Feb 27, 2001 at 09:23:33AM -0500 References: <01022706484400.20069@branagan> Message-ID: <20010227093012.C4181@tummy.com> On Tue, Feb 27, 2001 at 09:23:33AM -0500, Mark W. Alexander wrote: > (My understanding of) CPAN is that it's a repository of things that > install a standard way. You download what you want and > > perl Makefile.pl > make install CPAN is a lot closer to appget with debian than that. Yes, there is a standard perl Makefile, which looks through your system for quirks, and creates a makefile which can work. But it will also discover and resolve dependences in your perl site-packages, and automatically fetch them from your closest CPAN archive. Regards, Evelyn Mitchell efm@tummy.com From akuchlin@mems-exchange.org Tue Feb 27 11:43:12 2001 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Tue Feb 27 11:43:12 2001 Subject: [Distutils] Re: CPAN functionality for python - requirements In-Reply-To: <20010227093012.C4181@tummy.com>; from efm@tummy.com on Tue, Feb 27, 2001 at 09:30:13AM -0700 References: <01022706484400.20069@branagan> <20010227093012.C4181@tummy.com> Message-ID: <20010227114258.E17362@ute.cnri.reston.va.us> On Tue, Feb 27, 2001 at 09:30:13AM -0700, Evelyn Mitchell wrote: >On Tue, Feb 27, 2001 at 09:23:33AM -0500, Mark W. Alexander wrote: >> (My understanding of) CPAN is that it's a repository of things that >> install a standard way. You download what you want and >> >> perl Makefile.pl >> make install > >CPAN is a lot closer to appget with debian than that. >Yes, there is a standard perl Makefile, which looks through your >system for quirks, and creates a makefile which can work. Distutils aims to fix the installation problem, providing a consistent way to build a module once you've downloaded it. Discussion of indexing, dependencies, finding modules, &c., is the Catalog-SIG's job, and should be discussed there; they're off-topic for the Distutils SIG. --amk From sanner@scripps.edu Tue Feb 27 11:46:10 2001 From: sanner@scripps.edu (Michel Sanner) Date: Tue Feb 27 11:46:10 2001 Subject: [Distutils] Re: CPAN functionality for python - requirements In-Reply-To: Doug Hellmann "[Distutils] Re: CPAN functionality for python - requirements" (Feb 26, 10:45pm) References: <01022606501904.15046@branagan> <20010226183606.F7531@tummy.com> <01022623140707.15046@branagan> Message-ID: <1010227084403.ZM33944@noah.scripps.edu> Hi, I followed this discussion (probably not as much as I should have). We have setup a Python distribution site for the tools we have produced for computational biology. It is quite primitive but let's us distribute to "ready to go" packages as well as pre-compiled Python interpreters for some platforms to "non-wizard" users. One issue that we are strugling with is versions of packages.Is there a "standard" way to deal with versions ? how do you plan to find out whether someone has a compatible version of any given package ? -Michel On Feb 26, 10:45pm, Doug Hellmann wrote: > Subject: [Distutils] Re: CPAN functionality for python - requirements > On Mon, 26 Feb 2001, Sean Reifschneider wrote: > > On Mon, Feb 26, 2001 at 06:45:33AM -0500, Doug Hellmann wrote: > > >What if Python kept up with its own packages and modules like XEmacs does? We > > >could then use the same format on all platforms -- it might even be something > > >we roll ourself to make sure we can unpack it with *only* Python installed on a > > >platform. > > > > My understanding is that that's what distutils is meant to do, to an extent. > > The problem is that it's not entirely unreasonable to expect that an > > "rpm -qa" produce a list of all the software that's installed on your > > box. If you don't use an RPM, you can't really make use of that, and > > packages WILL get dropped after a re-install. > > The command "rpm -qa" only produces meaningful results on a system where RPM is > the default and primary package installation system. There are more OSes where > that is *not* the case (and where RPM isn't installed at all) than where it is. > > It still isn't clear to me why we would want to use more than one distribution > package format. It appears that one argument is so that sysadmins can use > those tools to track what is installed on the system. But isn't that part of > what this new thing (cyphon?) is supposed to do? Do we want Python to require > an external package management system on each platform where this tool is > supported, or do we want a Python-esq tool which just works the way it is > supposed to, in the same way, everywhere Python is available? > > But I think we're arguing over a design before we've worked out requirements. > The basic tasks of this new tool, as I see them, are: > > 1. Locate Python packages you want on the net. > 2. Resolve dependencies between what you want, what you have, and what you > need. > 3. Download the appropriate packages to give you what you want. > 4. Install those packages. > > Do we agree these are the primary features? > > I sense a consensus that the "install" part should be handled by distutils. Is > that right? > > Other requirements we might lay down up front: > > 1. Should run on all platforms where Python runs. > 2. Must support mirrors on the server side. > 3. Need to include documentation along with source for packages. > > The features related to dependencies and downloads could be handled by a > platform-specific packaging system, but integrating with all of the different > options on all of the different platforms where Python runs increases the > complexity of the new tool. (How do we handle RPMs on MacOS? How do we handle > HQX files on Win32? What about for Pippy, where files as such might not even > be appropriate? Does Jython have any special requirements?) Do we deal with > this complexity by deciding on a per-platform basis what format to use, or do > we force the user to specify the format they want downloaded? > > If we're modeling what we're doing based on CPAN, maybe we should look at > that design. What format does CPAN use when downloading Perl packages? What > features does CPAN have that we want? What does it not have that we want? > > Doug > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG@python.org > http://mail.python.org/mailman/listinfo/distutils-sig >-- End of excerpt from Doug Hellmann -- ----------------------------------------------------------------------- >>>>>>>>>> AREA CODE CHANGE <<<<<<<<< we are now 858 !!!!!!! Michel F. Sanner Ph.D. The Scripps Research Institute Assistant Professor Department of Molecular Biology 10550 North Torrey Pines Road Tel. (858) 784-2341 La Jolla, CA 92037 Fax. (858) 784-2860 sanner@scripps.edu http://www.scripps.edu/sanner ----------------------------------------------------------------------- From phrxy@csv.warwick.ac.uk Tue Feb 27 13:05:10 2001 From: phrxy@csv.warwick.ac.uk (John J. Lee) Date: Tue Feb 27 13:05:10 2001 Subject: [Catalog-sig] Re: [Distutils] Re: CPAN functionality for python - requirements In-Reply-To: Message-ID: On Tue, 27 Feb 2001, Mark W. Alexander wrote: [...] > As long as distutils can produce a binary for a particular platform/ > package manager combination, there's really no need for a dedicated > (official or UNofficial) "package mantainer". All that's needed > is a place where "python setup.py --bdist format=MyPackager" works. > How about an interarchive-API (or redirector?), so archives can > forward requests for binary packages to other sites known to support > that type of binary package? [...] In which case: 1. Distutils should be able to pgp-sign binary (and source as well come to think of it) packages. 2. CPyAN should be able to manage the signatures as appropriate. John From phrxy@csv.warwick.ac.uk Tue Feb 27 13:07:12 2001 From: phrxy@csv.warwick.ac.uk (John J. Lee) Date: Tue Feb 27 13:07:12 2001 Subject: [Distutils] Re: CPAN functionality for python In-Reply-To: Message-ID: On Mon, 26 Feb 2001, Tim Peters wrote: > [Guido] > > (So far my experience with Windows installers is much more > > positive than with RPMs though -- RPMs forever seem to depend > > on some version of some other RPM that you don't have.) > > [John J. Lee] > > Surely this is a function of free vs. non-free software as opposed to > > which particular packaging format you are using? > > I doubt it has much to do with either. It's more that, over very painful > years of installers that routinely crashed, Windows apps that rely on > anything external have learned to ship with *everything* they need. Heck, [...] Well yes, that was one of the things I was thinking of (I don't remember what the others were now :-). But you are right in saying that 'bundling everything' does not really divide along the free / non-free line. John From jafo@tummy.com Tue Feb 27 13:09:10 2001 From: jafo@tummy.com (Sean Reifschneider) Date: Tue Feb 27 13:09:10 2001 Subject: [Distutils] Re: CPAN functionality for python - requirements In-Reply-To: <20010227093012.C4181@tummy.com>; from efm@tummy.com on Tue, Feb 27, 2001 at 09:30:13AM -0700 References: <01022706484400.20069@branagan> <20010227093012.C4181@tummy.com> Message-ID: <20010227110803.Z1781@tummy.com> On Tue, Feb 27, 2001 at 09:30:13AM -0700, Evelyn Mitchell wrote: >But it will also discover and resolve dependences in your perl >site-packages, and automatically fetch them from your closest >CPAN archive. Not according to my tests the night before last. I did a test CPAN install of "News::Newsrc", which failed because the "make test" was failing. I then installed the "Set::BitSet" (? Something like that) module and tried News::Newsrc again and it worked... Maybe this was just a fluke and News::Newsrc is the exception and/or isn't used enough that people have gotten the prereqs right yet. If anyone knows for sure, I'm curious. Sean -- The dude abides. -- _The_Big_Lebowski_ Sean Reifschneider, Inimitably Superfluous tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python From phrxy@csv.warwick.ac.uk Tue Feb 27 13:17:11 2001 From: phrxy@csv.warwick.ac.uk (John J. Lee) Date: Tue Feb 27 13:17:11 2001 Subject: [Distutils] Re: CPAN functionality for python - requirements In-Reply-To: Message-ID: On Tue, 27 Feb 2001, Mark W. Alexander wrote: > On Tue, 27 Feb 2001, Doug Hellmann wrote: [...] > > I can't remember now whether CPAN did installations or just downloaded stuff > > and made you figure out how to install it. > > (My understanding of) CPAN is that it's a repository of things that > install a standard way. You download what you want and > > perl Makefile.pl > make install [...] There are two different 'CPAN's: 1. The archive itself -- a bunch of files in .tar.gz format, READMEs, etc. It uses a mirroring system that sets itself up on your first visit to point to a nearby mirror, and subsequently takes you there automatically. 2. The CPAN module, CPAN.pm, which can be used interactively or through its API. It lets you search for, download, find dependencies (and download again if required), build and install modules. [Activestate's windows distribution has its own binary package system, PPM, but CPAN (the module) works on windows as long as you have libwww-perl. I don't know what fraction of modules including C code compile without trouble on windows.] John From phrxy@csv.warwick.ac.uk Tue Feb 27 13:30:11 2001 From: phrxy@csv.warwick.ac.uk (John J. Lee) Date: Tue Feb 27 13:30:11 2001 Subject: [Distutils] Re: CPAN functionality for python - requirements In-Reply-To: <20010227110803.Z1781@tummy.com> Message-ID: Followup to Catalog-sig only (pine doesn't seem to let me set that in the headers...)? I don't think this has much to do with Distutils. On Tue, 27 Feb 2001, Sean Reifschneider wrote: > On Tue, Feb 27, 2001 at 09:30:13AM -0700, Evelyn Mitchell wrote: > >But it will also discover and resolve dependences in your perl > >site-packages, and automatically fetch them from your closest > >CPAN archive. > > Not according to my tests the night before last. I did a test CPAN > install of "News::Newsrc", which failed because the "make test" was > failing. I then installed the "Set::BitSet" (? Something like that) > module and tried News::Newsrc again and it worked... [...] Did you use CPAN.pm? CPAN the archive doesn't do dependencies. Dependencies are listed in the Makefile.PL (BTW, I have no idea why the capitalised PL), and after the module CPAN.pm downloads what you asked for, it checks that to see if there are dependencies, then downloads them if you don't already have them installed. John From jafo@tummy.com Tue Feb 27 14:17:12 2001 From: jafo@tummy.com (Sean Reifschneider) Date: Tue Feb 27 14:17:12 2001 Subject: [Distutils] Re: CPAN functionality for python - requirements In-Reply-To: <1010227084403.ZM33944@noah.scripps.edu>; from sanner@scripps.edu on Tue, Feb 27, 2001 at 08:44:02AM -0800 References: <01022606501904.15046@branagan> <20010226183606.F7531@tummy.com> <01022623140707.15046@branagan> <1010227084403.ZM33944@noah.scripps.edu> Message-ID: <20010227121607.D1781@tummy.com> On Tue, Feb 27, 2001 at 08:44:02AM -0800, Michel Sanner wrote: >One issue that we are strugling with is versions of packages.Is there a >"standard" way to deal with versions ? how do you plan to find out whether >someone has a compatible version of any given package ? That's something that I don't think we can do without human intervention. If package X version 1.50 "fixes" something, the dependencies will have to list "X >=1.20 <1.50". Something like that... In many cases it's safe to say "X >=1.20", but there are certainly some places where that's likely to have to be ammended on a case-by-case basis. Sean -- Gone Postal Sort: Iterate over elements, any element that is out of order you blow away. -- Evelyn, Kevin, and Sean, watching Monty Python and reading DDJ Sean Reifschneider, Inimitably Superfluous tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python From jafo@tummy.com Tue Feb 27 14:30:13 2001 From: jafo@tummy.com (Sean Reifschneider) Date: Tue Feb 27 14:30:13 2001 Subject: [Distutils] Re: CPAN functionality for python - requirements In-Reply-To: <01022706564001.20069@branagan>; from doughellmann@home.com on Tue, Feb 27, 2001 at 06:50:52AM -0500 References: <01022623140707.15046@branagan> <20010226214909.A10411@tummy.com> <01022706564001.20069@branagan> Message-ID: <20010227122903.F1781@tummy.com> On Tue, Feb 27, 2001 at 06:50:52AM -0500, Doug Hellmann wrote: >> That seems to be where we diverge... If somone has made an RPM of it, I'd >> rather have that than some files winding up hanging around my file-system. > >So why did distutils make it into the Python core? How does distutils making it into the Python distribution rule out that somone may want to prefer RPMs? >3. Need to support including documentation along with source for packages when >documentation is available. The practice in many RPMs is that the documentation is a "sub package" (see my Python SRPM), which can be installed on your development machines, but bypassed on the production machines... Don't get me wrong, I'm all for documentation. The lack of "javadoc" for Python has really hurt though... >But that leaves it up to the person posting the package to make distributions >in formats for any platforms where folks might want to install their stuff, or >to have someone *else* create those packages. That may lead to a situation What's wrong with allowing people to select different packaging formats if they're available? >where someone believes that they cannot install my package on MacOS because I >didn't make an HQX file (or whatever) when I posted HappyDoc. Again, my tool isn't concerned with such policies except that it allows the user to define it. If the user has defined a policy of "I won't accept any package that's not in .hqx format", then they should indeed not be shown HappyDoc if it doesn't have one. However, it would also allow them to say "I prefer .hqx files, but will take a distutils package if it's my only choice". That user then *WOULD* see HappyDoc if it had a distutils package. >I can go with "allow" but if we do not help then the user must specify in order >to get useful results from the tool, and that becomes "force." I don't follow that logic. >So, it sounds like their tool just does the download and then they use the >equivalent of distutils to do the installation. Is that all we want? For the archive server, I think it is... That doesn't preclude the archive client from being overly clever though... That's what I'm going to work on next, since I have the server in a state where it can handle giving out such information. Sean -- He who wonders discovers that this in itself is wonder. -- M. C. Escher Sean Reifschneider, Inimitably Superfluous tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python From jafo@tummy.com Tue Feb 27 14:54:12 2001 From: jafo@tummy.com (Sean Reifschneider) Date: Tue Feb 27 14:54:12 2001 Subject: [Distutils] Re: CPAN functionality for python - requirements In-Reply-To: <01022706484400.20069@branagan>; from doughellmann@home.com on Tue, Feb 27, 2001 at 06:28:32AM -0500 References: <01022706484400.20069@branagan> Message-ID: <20010227125351.H1781@tummy.com> On Tue, Feb 27, 2001 at 06:28:32AM -0500, Doug Hellmann wrote: >It sounds like the knows how to find a particular package, but getting it >installed is your problem. We can make it smarter than that. That's more a distutils problem than an archive network problem. My vision is that the archive network simply gives the client package information, which the client then can deal with as they see fit. Ideally, once downloaded, the archive network client would then just tell the package "Install thyself", but there will probably have to be some additional smarts in the client than that. I expect these additional smarts to be little more than scripts: mkdir tmpdirectory cd tmpdirectory tar xpvfz archivefile cd archivename-with-.tar.gz-removed python setup.py That sort of thing... >you make the cyphon client download the right package format or convert >a standard distro into the right format? Doesn't the user have to specify a >package type? That decision is up to the client. If the client has the smarts to turn a distutils package into an RPM, then the client would list it's preferred format as a distutils package and it would handle the rest. If it can't, you can either select an RPM, or fall back to a distutils package. >No, it isn't. Even if *all* the tool does is download things I told it to find >and write them to the current directory, it has to know what format I like or >it may download (for example) debian packages for my Mac -- and then I won't >get useful results from the tool. That's exactly my target. >> Complexity is only increased if the new tool thinks it should handle >> platform/OS/distro specific issues. > >It *must* handle those issues to be useful. The schema lists platform version, name, architecture, and format. So, it can handle "RedHat 7.0 ix86 in RPM format" if we wish. Any of those fields can be "*", which will probably be the case in source distutils package cases. >Wait -- maybe that's where the disconnect is! I'm describing features of the >system from the client-side view, assuming the network would provide services >to support those features. It sounds like you are describing features of the The archive network client I believe should have a step which is "install this", but I believe that largely needs to be abstracted from the client. Scripts that are used to control the "I've got this thing, here's how to install it". In the case of a tar file (not a distutils package) we probably want to extract it then drop the user into a shell saying "Check out the README and INSTALL files, do the install, then type 'exit' when done". Automating it to the point that we can... >network. We should determine the requirements from the user perspective first, >and design something to meet those requirements. Designs for the network >services will follow from the requirements. That's why I've been posting my schema and trying to decide what my vision of the client interaction is. If it doesn't meet the user needs, then it needs to be re-worked. However, I believe it has the flexability to handle the policy decisions that have been discussed here. >I can't remember now whether CPAN did installations or just downloaded stuff >and made you figure out how to install it. It does a "make test" and a "make install". There is a convention that CPAN packages will have a "make test", even if it doesn't do much. It's a great idea... With Python modules we could almost make some of the dependencies automatic. If we try to import the module and it throws an exception on an import, we know we've hit a dependency... That'd be cute... >Right, so we need a single way of determining, maintaining, and >resolving dependencies. One could say that the appropriate way to find out if a required module is available is to try importing it. I firmly believe that we have to allow not only system-wide installs, but also user installs (into their home directory, for example). Sean -- Come see the violence inherent in the system! Help! Help! I'm being repressed! -- Monty Python and the Holy Grail Sean Reifschneider, Inimitably Superfluous tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python From jafo@tummy.com Tue Feb 27 15:21:12 2001 From: jafo@tummy.com (Sean Reifschneider) Date: Tue Feb 27 15:21:12 2001 Subject: [Distutils] Re: CPAN functionality for python - requirements In-Reply-To: ; from bsass@freenet.edmonton.ab.ca on Tue, Feb 27, 2001 at 02:49:40AM -0700 References: <20010226214909.A10411@tummy.com> Message-ID: <20010227132028.L1781@tummy.com> On Tue, Feb 27, 2001 at 02:49:40AM -0700, Bruce Sass wrote: >Even if the RPM was made by someone who didn't have a clue? Yes, because it's easier to remove an RPM from somone who doesn't have a clue than a .tar file from somone who doesn't have a clue. >Wouldn't it be better to have a standard way to convert a Python >package into an RPM, that way any packaging problems will be seen by >everyone and the fix will be fast. The thing about automated systems is that they often fail in particularly spetacular ways. If that happens an human intervention is required, then the archive network should be able to handle it, without requiring that distutils be fixed, then handed off to the author of the package to make a new release from. >Huh, there are tools and standards. >Isn't /usr/shar/doc/package the FHS place for general package specific Where are the tools for turning doc-strings into HTML, and how are the links between doc packages handled? What is the markup format for the doc strings? We spent quite a while discussing the sad state of the documentation at our last pythoneers meeting. Mark Lutz was shocked that people would actually use javadoc, but they do seem to simply becuase it was available and was "good enough". We are missing similar functionality. Sean -- Give me immortality or give me death! Sean Reifschneider, Inimitably Superfluous tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python From bsass@freenet.edmonton.ab.ca Tue Feb 27 17:40:02 2001 From: bsass@freenet.edmonton.ab.ca (Bruce Sass) Date: Tue Feb 27 17:40:02 2001 Subject: [Distutils] Re: CPAN functionality for python - requirements In-Reply-To: <20010227125351.H1781@tummy.com> Message-ID: I'm a little concerned about the privacy aspect of this... > That decision is up to the client. If the client has the smarts to turn > a distutils package into an RPM, then the client would list it's preferred > format as a distutils package and it would handle the rest. If it can't, > you can either select an RPM, or fall back to a distutils package. Why should the client need to "list" anything. client: what do you have? server: this(deb,rh-5.rpm,rh-6.2.rpm,rh-7.rpm) that() other(hqx) client: send me this(deb), that(mandrake.rpm), other() server: sorry don't have that(mandrake.rpm) client: send me that(deb) server: sorry don't have that(deb) client: send me that() This is how I would handle a curious server... client: what do you have? server: what format do you want? In other words... the server is just that - a server - it does not ask questions, it does not keep track of any preferences, it serves up files. This could work... client: only show me (deb) and () client: what do you have? ... The server should NOT be usable as a tool to track Python users and their habits, and making it do so should require a conscious effort on the operators part (so there is no opportunity for the operator to say, "I don't track you, that's just how the software works"). - Bruce From Anthony@COMPUTRONIX.com Tue Feb 27 18:43:01 2001 From: Anthony@COMPUTRONIX.com (Anthony Tuininga) Date: Tue Feb 27 18:43:01 2001 Subject: [Distutils] Potential patch for sysconfig.py Message-ID: This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C0A116.ED5C1040 Content-Type: text/plain; charset="iso-8859-1" I added these lines to _int_nt() to support creating extensions under NT with gcc (Mingw32). I don't know a whole lot about distutils as of yet but these lines worked for me. Any comments? g['CC'] = "gcc" g['OPT'] = "" g['CCSHARED'] = "" g['LDSHARED'] = "dllwrap" Anthony Tuininga ------_=_NextPart_001_01C0A116.ED5C1040 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Potential patch for sysconfig.py

I added these lines to _int_nt() to = support creating extensions under NT with gcc (Mingw32). I don't know a = whole lot about distutils as of yet but these lines worked for me. Any = comments?

    g['CC'] =3D = "gcc"
    g['OPT'] =3D = ""
    g['CCSHARED'] =3D = ""
    g['LDSHARED'] =3D = "dllwrap"

Anthony Tuininga

------_=_NextPart_001_01C0A116.ED5C1040-- From doughellmann@home.com Tue Feb 27 21:10:12 2001 From: doughellmann@home.com (Doug Hellmann) Date: Tue Feb 27 21:10:12 2001 Subject: [Distutils] Re: CPAN functionality for python - requirements In-Reply-To: References: Message-ID: <01022721090603.20069@branagan> On Tue, 27 Feb 2001, Bruce Sass wrote: > I'm a little concerned about the privacy aspect of this... > > > That decision is up to the client. If the client has the smarts to turn > > a distutils package into an RPM, then the client would list it's preferred > > format as a distutils package and it would handle the rest. If it can't, > > you can either select an RPM, or fall back to a distutils package. > > Why should the client need to "list" anything. > > client: what do you have? > server: this(deb,rh-5.rpm,rh-6.2.rpm,rh-7.rpm) that() other(hqx) > client: send me this(deb), that(mandrake.rpm), other() > server: sorry don't have that(mandrake.rpm) > client: send me that(deb) > server: sorry don't have that(deb) > client: send me that() I'm not sure why the "what do you have" question is needed. The "send me that(mandrake.rpm)" interaction is what we want. The client requests packages in the formats prefered. A standard source format should be available for all packages so that everybody can get every package in some form. If the standard source format contains distutil-enabled scripts, platform specific distribution files could be generated from the source. > In other words... the server is just that - a server - it does not ask > questions, it does not keep track of any preferences, it serves up > files. No, of course not. > This could work... > client: only show me (deb) and () > client: what do you have? Right, that would be a bit more efficient than specifying a format for every package for every request. > The server should NOT be usable as a tool to track Python users and > their habits, and making it do so should require a conscious effort on > the operators part (so there is no opportunity for the operator to > say, "I don't track you, that's just how the software works"). The server is likely to be a cgi, which by its nature means requests may be written to a log file. Should that be disallowed? Doug From jafo@tummy.com Tue Feb 27 23:01:13 2001 From: jafo@tummy.com (Sean Reifschneider) Date: Tue Feb 27 23:01:13 2001 Subject: [Distutils] Re: CPAN functionality for python - requirements In-Reply-To: <01022721090603.20069@branagan>; from doughellmann@home.com on Tue, Feb 27, 2001 at 09:03:43PM -0500 References: <01022721090603.20069@branagan> Message-ID: <20010227204138.C1781@tummy.com> On Tue, Feb 27, 2001 at 09:03:43PM -0500, Doug Hellmann wrote: >> > That decision is up to the client. If the client has the smarts to turn >> > a distutils package into an RPM, then the client would list it's preferred >> > format as a distutils package and it would handle the rest. If it can't, >> > you can either select an RPM, or fall back to a distutils package. >> >> Why should the client need to "list" anything. If you mean "why would it need to send the list of preferred package formats to the server", it would do that so that the server could return the list of specific entries that the client wants. Currently, the implementation is that the client can ask for a specific package format, and can ask again if there isn't one available, or it can ask the server to list all of them that it has, and then discard the ones it does't want. If this is a privacy concern, don't use it, eh? We can only do so much if you want to be tight-lipped: "I want a package, but I don't want to say what it is." "Ok." >I'm not sure why the "what do you have" question is needed. The "send me >that(mandrake.rpm)" interaction is what we want. If you know exactly the versions, package formats, etc which are available? One could say that if you expect the client to know all this, then you might as well expect them to know where to get the thing... >The server is likely to be a cgi, which by its nature means requests may be >written to a log file. Should that be disallowed? A user is more than welcome to use an anonimizer for connecting to the archive. Sean -- A computer is like an Old Testament god, with a lot of rules and no mercy. -- Joseph Campbell Sean Reifschneider, Inimitably Superfluous tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python From jafo@tummy.com Tue Feb 27 23:04:04 2001 From: jafo@tummy.com (Sean Reifschneider) Date: Tue Feb 27 23:04:04 2001 Subject: [Distutils] Re: CPAN functionality for python - requirements In-Reply-To: <01022721090603.20069@branagan>; from doughellmann@home.com on Tue, Feb 27, 2001 at 09:03:43PM -0500 References: <01022721090603.20069@branagan> Message-ID: <20010227204138.C1781@tummy.com> On Tue, Feb 27, 2001 at 09:03:43PM -0500, Doug Hellmann wrote: >> > That decision is up to the client. If the client has the smarts to turn >> > a distutils package into an RPM, then the client would list it's preferred >> > format as a distutils package and it would handle the rest. If it can't, >> > you can either select an RPM, or fall back to a distutils package. >> >> Why should the client need to "list" anything. If you mean "why would it need to send the list of preferred package formats to the server", it would do that so that the server could return the list of specific entries that the client wants. Currently, the implementation is that the client can ask for a specific package format, and can ask again if there isn't one available, or it can ask the server to list all of them that it has, and then discard the ones it does't want. If this is a privacy concern, don't use it, eh? We can only do so much if you want to be tight-lipped: "I want a package, but I don't want to say what it is." "Ok." >I'm not sure why the "what do you have" question is needed. The "send me >that(mandrake.rpm)" interaction is what we want. If you know exactly the versions, package formats, etc which are available? One could say that if you expect the client to know all this, then you might as well expect them to know where to get the thing... >The server is likely to be a cgi, which by its nature means requests may be >written to a log file. Should that be disallowed? A user is more than welcome to use an anonimizer for connecting to the archive. Sean -- A computer is like an Old Testament god, with a lot of rules and no mercy. -- Joseph Campbell Sean Reifschneider, Inimitably Superfluous tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python -- http://mail.python.org/mailman/listinfo/python-list From jafo@tummy.com Tue Feb 27 23:14:01 2001 From: jafo@tummy.com (Sean Reifschneider) Date: Tue Feb 27 23:14:01 2001 Subject: [Distutils] Re: CPAN functionality for python - requirements In-Reply-To: <01022710324002.20069@branagan>; from doughellmann@home.com on Tue, Feb 27, 2001 at 10:17:06AM -0500 References: <01022710324002.20069@branagan> Message-ID: <20010227211331.A24378@tummy.com> On Tue, Feb 27, 2001 at 10:17:06AM -0500, Doug Hellmann wrote: >OR automatically. If the source distribution is always available, part of the >mirroring process could be to convert the source distributions into binary >distributions for the platform serviced by the mirror site. That seems like it would be a huge security issue. Sean -- Passionate hatred can give meaning and purpose to an empty life. -- Eric Hoffer Sean Reifschneider, Inimitably Superfluous tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python From jafo@tummy.com Tue Feb 27 23:16:11 2001 From: jafo@tummy.com (Sean Reifschneider) Date: Tue Feb 27 23:16:11 2001 Subject: [Catalog-sig] Re: [Distutils] Re: CPAN functionality for python - requirements In-Reply-To: ; from mwa@gate.net on Tue, Feb 27, 2001 at 10:55:03AM -0500 References: <01022710324002.20069@branagan> Message-ID: <20010227211525.B24378@tummy.com> On Tue, Feb 27, 2001 at 10:55:03AM -0500, Mark W. Alexander wrote: >That was my original thought. I would hate to see, however, that some >platform gets "slighted" because there's no one to run a mirror for >them. It seems to me that instead of having the "mirrors" do it, some work be done on setting up "build nodes" that can take new source packages and turn them into binaries... Sean -- It often shows a fine command of a language to say nothing. Sean Reifschneider, Inimitably Superfluous tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python From jafo@tummy.com Tue Feb 27 23:18:03 2001 From: jafo@tummy.com (Sean Reifschneider) Date: Tue Feb 27 23:18:03 2001 Subject: [Distutils] Re: CPAN functionality for python - requirements In-Reply-To: <01022710324002.20069@branagan>; from doughellmann@home.com on Tue, Feb 27, 2001 at 10:17:06AM -0500 References: <01022710324002.20069@branagan> Message-ID: <20010227211331.A24378@tummy.com> On Tue, Feb 27, 2001 at 10:17:06AM -0500, Doug Hellmann wrote: >OR automatically. If the source distribution is always available, part of the >mirroring process could be to convert the source distributions into binary >distributions for the platform serviced by the mirror site. That seems like it would be a huge security issue. Sean -- Passionate hatred can give meaning and purpose to an empty life. -- Eric Hoffer Sean Reifschneider, Inimitably Superfluous tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python -- http://mail.python.org/mailman/listinfo/python-list From jafo@tummy.com Tue Feb 27 23:19:07 2001 From: jafo@tummy.com (Sean Reifschneider) Date: Tue Feb 27 23:19:07 2001 Subject: [Catalog-sig] Re: [Distutils] Re: CPAN functionality for python - requirements In-Reply-To: ; from mwa@gate.net on Tue, Feb 27, 2001 at 10:55:03AM -0500 References: <01022710324002.20069@branagan> Message-ID: <20010227211525.B24378@tummy.com> On Tue, Feb 27, 2001 at 10:55:03AM -0500, Mark W. Alexander wrote: >That was my original thought. I would hate to see, however, that some >platform gets "slighted" because there's no one to run a mirror for >them. It seems to me that instead of having the "mirrors" do it, some work be done on setting up "build nodes" that can take new source packages and turn them into binaries... Sean -- It often shows a fine command of a language to say nothing. Sean Reifschneider, Inimitably Superfluous tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python -- http://mail.python.org/mailman/listinfo/python-list From R.Liebscher@gmx.de Wed Feb 28 03:09:01 2001 From: R.Liebscher@gmx.de (Rene Liebscher) Date: Wed Feb 28 03:09:01 2001 Subject: [Distutils] Potential patch for sysconfig.py References: Message-ID: <3A9CB1E9.2BC6BDE5@gmx.de> Anthony Tuininga wrote: > > Part 1.1Type: Plain Text (text/plain) > I added these lines to _int_nt() to support creating extensions under NT > with gcc (Mingw32). I don't know a whole lot about distutils as of yet but > these lines worked for me. Any comments? > > g['CC'] = "gcc" > g['OPT'] = "" > g['CCSHARED'] = "" > g['LDSHARED'] = "dllwrap" > > Anthony Tuininga > Did you ever try "python setup.py build --compiler=mingw32" with your package? Mingw32 and cygwin are supported for more than a half year now. (Ok, we should have updated the documentation, but to use help with the build command isn't a problem?) Kind regards Rene Liebscher From jack@oratrix.nl Wed Feb 28 04:15:00 2001 From: jack@oratrix.nl (Jack Jansen) Date: Wed Feb 28 04:15:00 2001 Subject: [Distutils] Re: CPAN functionality for python - requirements In-Reply-To: Message by Doug Hellmann , Tue, 27 Feb 2001 21:03:43 -0500 , <01022721090603.20069@branagan> Message-ID: <20010228091351.C4A41371690@snelboot.oratrix.nl> > I'm not sure why the "what do you have" question is needed. The "send me > that(mandrake.rpm)" interaction is what we want. Well, one of the things I'd like to do is integrate all this in MacPython IDE, and the same probably holds for PythonWin and Idle. Then the "what do you have" is pretty useful for building menus and such. -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm From jack@oratrix.nl Wed Feb 28 04:20:01 2001 From: jack@oratrix.nl (Jack Jansen) Date: Wed Feb 28 04:20:01 2001 Subject: [Distutils] Re: CPAN functionality for python - requirements In-Reply-To: Message by Doug Hellmann , Tue, 27 Feb 2001 21:03:43 -0500 , <01022721090603.20069@branagan> Message-ID: <20010228091351.C4A41371690@snelboot.oratrix.nl> > I'm not sure why the "what do you have" question is needed. The "send me > that(mandrake.rpm)" interaction is what we want. Well, one of the things I'd like to do is integrate all this in MacPython IDE, and the same probably holds for PythonWin and Idle. Then the "what do you have" is pretty useful for building menus and such. -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm -- http://mail.python.org/mailman/listinfo/python-list From Paul.Moore@uk.origin-it.com Wed Feb 28 05:00:14 2001 From: Paul.Moore@uk.origin-it.com (Moore, Paul) Date: Wed Feb 28 05:00:14 2001 Subject: [Distutils] Re: CPAN functionality for python - requirements Message-ID: <714DFA46B9BBD0119CD000805FC1F53B01B5AD44@UKRUX002.rundc.uk.origin-it.com> From: Sean Reifschneider [mailto:jafo@tummy.com] > On Tue, Feb 27, 2001 at 09:30:13AM -0700, Evelyn Mitchell wrote: > >But it will also discover and resolve dependences in your perl > >site-packages, and automatically fetch them from your closest > >CPAN archive. > > Not according to my tests the night before last. I did a test CPAN > install of "News::Newsrc", which failed because the "make test" was > failing. I then installed the "Set::BitSet" (? Something like that) > module and tried News::Newsrc again and it worked... > > Maybe this was just a fluke and News::Newsrc is the exception and/or > isn't used enough that people have gotten the prereqs right yet. If > anyone knows for sure, I'm curious. There are basically a number of aspects to "CPAN", which need separating out. MakeMaker --------- This is a perl module which implements a build process. You write a Makefile.PL, which calls MakeMaker defining the module's structure and metadata. Thanks to MakeMaker, the process for building a Perl module is (nearly always) simply perl Makefile.PL make make test <-- optional, but pretty much standard - runs module unit tests make install <-- installs the module This is, in both concept and functionality, almost identical to Distutils. There are some areas in which distutils is better (building of platform-specific installers, for instance) and some where MakeMaker is better (it hooks into a very standard structure for modules, generated by thye h2xs program, which practically forces module authors to write test suites and documentation in a standard format), but these are details. We can consider this covered. (Although the distutils-sig could still usefully learn from MakeMaker). The system of FTP sites and mirrors ----------------------------------- Frankly, this is nothing special. It has some nice features (automated uploads for module authors, plus quite nice indexing and server multiplexing features), but it isn't rocket science as far as I know. We could quite happily start with a simple FTP site for this (that's what CPAN did - the mirroring came later as popularity grew). CPAN.pm ------- This is a Perl module, which automates the process of downloading and installing modules. I don't use this personally, for a number of reasons. Frankly, I find that manual downloading and running the 4 lines noted above is fine. It relies for things like dependency tracking on metadata added into the Makefile.PL which is not necessary for the straight 4-line build above. As such, the level to which that metadata is added by module authors is variable (to be polite). In practice, I wouldn't rely on it - accurate dependency data seems to be the exception rather than the rule. I *don't* regard CPAN.pm to be important to the overall CPAN "phenomenon". But some people like it. Writing something "at least as good as" CPAN.pm shouldn't be hard in Python - not least because the standard library is rich enough that things like FTP client support is available out of the box (whereas CPAN.pm won't work until you manually install libnet, and possibly some other modules, I forget which...) But writing a "perfect" utility for automated download-and-install, with dependency tracking, etc etc, is going to be VERY HARD. Don't get misled - Perl doesn't have such a beast. And We won't even have what Perl has if we focus on perfection rather than practicality. The h2xs program ---------------- This is VERY important. The idea is that when you write a Perl module, either pure perl or a C (XS) extension, you run h2xs first, to generate a template build directory. It automatically includes * The perl module, with some basic template code and embedded POD documentation * The XS extension, with template code (if requested) * A Makefile.PL shell * A basic test script - all it does is test that the module loads, but it includes a placeholder for your own tests Essentially, h2xs forces a standard structure on all Perl modules. This is important for Perl, where modules have to conform to some standards in order to work at all. However, it brings HUGE benefits in standardisation of all the "other" parts of the process (documentation, tests, etc). Python is at a disadvantage here, precisely because writing a Python module involves so little in the way of a specific structure. So people will likely rebel against having a structure "imposed"... A social structure ------------------ This is a bit of a chicken and egg issue, but Perl developers expect to write modules using h2xs and MakeMaker, they expect to write tests (even if they are minimal), they expect to fill in the sections in the POD documentation, and they expect to submit their modules to CPAN. So this all "just works". Interestingly, developers probably don't "expect" to have to include dependency information, and hence many don't - resulting in the problems you hit. But then again, Perl users don't "expect" to be totally shielded from dependency issues. Python is VERY far behind here. This is a maturity issue - distutils is still (relatively) new, and so there are LOTS of packages which don't come with a setup.py yet. Often, adding one isn't hard, but it is still to happen. And when you are distributing a pure python module, as a single .py file, it's hard to see the benefit of changing that into a .tar.gz file containing the module, plus a setup.py. (Once you start adding test suites and documentation, the point of the whole archive bit is clearer, but we're not even close to that stage yet). Things are looking better with Python 2.1, though. Included with 2.1, it looks like there will be a standard unit testing module, and a new "pydoc" package, which will extract documentation from module docstrings. So the infrastructure will be there, it just becomes a case of getting developers to use it consistently. Distutils can help with this, by "just working" if people follow a standard structure. Sorry, this wasn't meant to get so long... Paul. From Paul.Moore@uk.origin-it.com Wed Feb 28 05:03:01 2001 From: Paul.Moore@uk.origin-it.com (Moore, Paul) Date: Wed Feb 28 05:03:01 2001 Subject: [Distutils] Re: CPAN functionality for python - requirements Message-ID: <714DFA46B9BBD0119CD000805FC1F53B01B5AD44@UKRUX002.rundc.uk.origin-it.com> From: Sean Reifschneider [mailto:jafo@tummy.com] > On Tue, Feb 27, 2001 at 09:30:13AM -0700, Evelyn Mitchell wrote: > >But it will also discover and resolve dependences in your perl > >site-packages, and automatically fetch them from your closest > >CPAN archive. > > Not according to my tests the night before last. I did a test CPAN > install of "News::Newsrc", which failed because the "make test" was > failing. I then installed the "Set::BitSet" (? Something like that) > module and tried News::Newsrc again and it worked... > > Maybe this was just a fluke and News::Newsrc is the exception and/or > isn't used enough that people have gotten the prereqs right yet. If > anyone knows for sure, I'm curious. There are basically a number of aspects to "CPAN", which need separating out. MakeMaker --------- This is a perl module which implements a build process. You write a Makefile.PL, which calls MakeMaker defining the module's structure and metadata. Thanks to MakeMaker, the process for building a Perl module is (nearly always) simply perl Makefile.PL make make test <-- optional, but pretty much standard - runs module unit tests make install <-- installs the module This is, in both concept and functionality, almost identical to Distutils. There are some areas in which distutils is better (building of platform-specific installers, for instance) and some where MakeMaker is better (it hooks into a very standard structure for modules, generated by thye h2xs program, which practically forces module authors to write test suites and documentation in a standard format), but these are details. We can consider this covered. (Although the distutils-sig could still usefully learn from MakeMaker). The system of FTP sites and mirrors ----------------------------------- Frankly, this is nothing special. It has some nice features (automated uploads for module authors, plus quite nice indexing and server multiplexing features), but it isn't rocket science as far as I know. We could quite happily start with a simple FTP site for this (that's what CPAN did - the mirroring came later as popularity grew). CPAN.pm ------- This is a Perl module, which automates the process of downloading and installing modules. I don't use this personally, for a number of reasons. Frankly, I find that manual downloading and running the 4 lines noted above is fine. It relies for things like dependency tracking on metadata added into the Makefile.PL which is not necessary for the straight 4-line build above. As such, the level to which that metadata is added by module authors is variable (to be polite). In practice, I wouldn't rely on it - accurate dependency data seems to be the exception rather than the rule. I *don't* regard CPAN.pm to be important to the overall CPAN "phenomenon". But some people like it. Writing something "at least as good as" CPAN.pm shouldn't be hard in Python - not least because the standard library is rich enough that things like FTP client support is available out of the box (whereas CPAN.pm won't work until you manually install libnet, and possibly some other modules, I forget which...) But writing a "perfect" utility for automated download-and-install, with dependency tracking, etc etc, is going to be VERY HARD. Don't get misled - Perl doesn't have such a beast. And We won't even have what Perl has if we focus on perfection rather than practicality. The h2xs program ---------------- This is VERY important. The idea is that when you write a Perl module, either pure perl or a C (XS) extension, you run h2xs first, to generate a template build directory. It automatically includes * The perl module, with some basic template code and embedded POD documentation * The XS extension, with template code (if requested) * A Makefile.PL shell * A basic test script - all it does is test that the module loads, but it includes a placeholder for your own tests Essentially, h2xs forces a standard structure on all Perl modules. This is important for Perl, where modules have to conform to some standards in order to work at all. However, it brings HUGE benefits in standardisation of all the "other" parts of the process (documentation, tests, etc). Python is at a disadvantage here, precisely because writing a Python module involves so little in the way of a specific structure. So people will likely rebel against having a structure "imposed"... A social structure ------------------ This is a bit of a chicken and egg issue, but Perl developers expect to write modules using h2xs and MakeMaker, they expect to write tests (even if they are minimal), they expect to fill in the sections in the POD documentation, and they expect to submit their modules to CPAN. So this all "just works". Interestingly, developers probably don't "expect" to have to include dependency information, and hence many don't - resulting in the problems you hit. But then again, Perl users don't "expect" to be totally shielded from dependency issues. Python is VERY far behind here. This is a maturity issue - distutils is still (relatively) new, and so there are LOTS of packages which don't come with a setup.py yet. Often, adding one isn't hard, but it is still to happen. And when you are distributing a pure python module, as a single .py file, it's hard to see the benefit of changing that into a .tar.gz file containing the module, plus a setup.py. (Once you start adding test suites and documentation, the point of the whole archive bit is clearer, but we're not even close to that stage yet). Things are looking better with Python 2.1, though. Included with 2.1, it looks like there will be a standard unit testing module, and a new "pydoc" package, which will extract documentation from module docstrings. So the infrastructure will be there, it just becomes a case of getting developers to use it consistently. Distutils can help with this, by "just working" if people follow a standard structure. Sorry, this wasn't meant to get so long... Paul. -- http://mail.python.org/mailman/listinfo/python-list From Paul.Moore@uk.origin-it.com Wed Feb 28 05:09:00 2001 From: Paul.Moore@uk.origin-it.com (Moore, Paul) Date: Wed Feb 28 05:09:00 2001 Subject: [Distutils] Distutils suggestion - test action Message-ID: <714DFA46B9BBD0119CD000805FC1F53B01B5AD45@UKRUX002.rundc.uk.origin-it.com> Following on from my message about "CPAN", I was thinking that one extremely useful addition to distutils would be to add a "test" action. The basic approach would be that python setup.py test would run a test suite. Of course, for this to work, it would require modules to put tests in "a standard place". There are two main options - the first is for modules to have tests inline, via if __name__=="__main__". While this is common, I would suggest instead that running a separate test.py would be better. There's nothing stopping the developer having inline tests, it just means that test.py becomes a simple wrapper (execfile("module.py")). If test.py does not exist, distutils should print a simple message "No tests defined". Do people think this is a good idea? I would be willing to look into implementing this, although I have no experience in hacking distutils. Paul. From akuchlin@mems-exchange.org Wed Feb 28 07:50:01 2001 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Wed Feb 28 07:50:01 2001 Subject: [Distutils] Distutils suggestion - test action In-Reply-To: <714DFA46B9BBD0119CD000805FC1F53B01B5AD45@UKRUX002.rundc.uk.origin-it.com>; from Paul.Moore@uk.origin-it.com on Wed, Feb 28, 2001 at 10:08:13AM -0000 References: <714DFA46B9BBD0119CD000805FC1F53B01B5AD45@UKRUX002.rundc.uk.origin-it.com> Message-ID: <20010228074901.A17704@newcnri.cnri.reston.va.us> On Wed, Feb 28, 2001 at 10:08:13AM -0000, Moore, Paul wrote: >test.py would be better. There's nothing stopping the developer having >inline tests, it just means that test.py becomes a simple wrapper >(execfile("module.py")). Another approach would be to have a test/ subdirectory, and scan through all the *.py files in it (all the test*.py files? should it recursively scan for test/ directories through a large project?). Can you run just a single test? Another issue: how do tests signal their success or failure to the 'test' command? Writing a Distutils command isn't too hard; read the class docstring in distutils/cmd.py for info. --amk From Anthony@COMPUTRONIX.com Wed Feb 28 09:48:01 2001 From: Anthony@COMPUTRONIX.com (Anthony Tuininga) Date: Wed Feb 28 09:48:01 2001 Subject: [Distutils] Potential patch for sysconfig.py Message-ID: This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C0A195.5A5F54D0 Content-Type: text/plain; charset="iso-8859-1" Ok, I looked at the help but missed the line --help-compiler which gave the list of compilers. I have now tried --compiler=mingw32 as suggested and that works quite well. The documentation inside the code doesn't mention anything about compilers except unix and msvc either -- or at least the part I looked at.... :-) In any case, thanks for your help! Anthony -----Original Message----- From: Rene Liebscher [mailto:R.Liebscher@gmx.de] Sent: Wednesday, February 28, 2001 1:08 AM To: Anthony Tuininga Cc: distutils-sig@python.org Subject: Re: [Distutils] Potential patch for sysconfig.py Anthony Tuininga wrote: > > Part 1.1Type: Plain Text (text/plain) > I added these lines to _int_nt() to support creating extensions under NT > with gcc (Mingw32). I don't know a whole lot about distutils as of yet but > these lines worked for me. Any comments? > > g['CC'] = "gcc" > g['OPT'] = "" > g['CCSHARED'] = "" > g['LDSHARED'] = "dllwrap" > > Anthony Tuininga > Did you ever try "python setup.py build --compiler=mingw32" with your package? Mingw32 and cygwin are supported for more than a half year now. (Ok, we should have updated the documentation, but to use help with the build command isn't a problem?) Kind regards Rene Liebscher ------_=_NextPart_001_01C0A195.5A5F54D0 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable RE: [Distutils] Potential patch for sysconfig.py

Ok, I looked at the help but missed the line = --help-compiler which gave the list of compilers. I have now tried = --compiler=3Dmingw32 as suggested and that works quite well. The = documentation inside the code doesn't mention anything about compilers = except unix and msvc either -- or at least the part I looked at.... :-) = In any case, thanks for your help!

Anthony

-----Original Message-----
From: Rene Liebscher [mailto:R.Liebscher@gmx.de]=
Sent: Wednesday, February 28, 2001 1:08 AM
To: Anthony Tuininga
Cc: distutils-sig@python.org
Subject: Re: [Distutils] Potential patch for = sysconfig.py


Anthony Tuininga wrote:
>
>    Part 1.1Type: Plain Text = (text/plain)
> I added these lines to _int_nt() to support = creating extensions under NT
> with gcc (Mingw32). I don't know a whole lot = about distutils as of yet but
> these lines worked for me. Any comments?
>
>     g['CC'] =3D = "gcc"
>     g['OPT'] =3D = ""
>     g['CCSHARED'] =3D = ""
>     g['LDSHARED'] =3D = "dllwrap"
>
> Anthony Tuininga
>

Did you ever try "python setup.py build = --compiler=3Dmingw32"
with your package?

Mingw32 and cygwin are supported for more than a half = year now.

(Ok, we should have updated the documentation, but to = use
help with the build command isn't a problem?)  =


Kind regards
Rene Liebscher

------_=_NextPart_001_01C0A195.5A5F54D0-- From guido@digicool.com Wed Feb 28 10:29:00 2001 From: guido@digicool.com (Guido van Rossum) Date: Wed Feb 28 10:29:00 2001 Subject: [Distutils] Please disable your mail mirror, whoever it is! In-Reply-To: Your message of "28 Feb 2001 01:17:57 PST." <20010228091351.C4A41371690@snelboot.oratrix.nl> References: <20010228091351.C4A41371690@snelboot.oratrix.nl> Message-ID: <200102281526.KAA28444@cj20424-a.reston1.va.home.com> Somebody or something is echoing all mail in this thread. Below is a sample message -- look at all the repeated headers with an X- in front of them. Can this please stop? --Guido van Rossum (home page: http://www.python.org/~guido/) > Return-Path: distutils-sig-admin@python.org > Delivery-Date: Wed Feb 28 04:25:40 2001 > Return-Path: > Received: from localhost (IDENT:guido@localhost.localdomain [127.0.0.1]) > by cj20424-a.reston1.va.home.com (8.9.3/8.9.3) with ESMTP id EAA23220 > for ; Wed, 28 Feb 2001 04:25:40 -0500 > Received: from pop.digicool.com > by localhost with POP3 (fetchmail-5.3.1) > for guido@localhost (single-drop); Wed, 28 Feb 2001 04:25:40 -0500 (EST) > Received: from ns2.digicool.com ([216.164.72.2] verified) > by digicool.com (CommuniGate Pro SMTP 3.4) > with ESMTP id 1628614 for guido@mail.digicool.com; Wed, 28 Feb 2001 04:24:25 -0500 > Received: from mail.python.org (mail.python.org [63.102.49.29]) > by ns2.digicool.com (8.9.3/8.9.3) with ESMTP id EAA06781 > for ; Wed, 28 Feb 2001 04:22:01 -0500 > Received: from localhost.localdomain ([127.0.0.1] helo=mail.python.org) > by mail.python.org with esmtp (Exim 3.21 #1) > id 14Y2mB-0003d5-00; Wed, 28 Feb 2001 04:20:03 -0500 > Received: from [204.123.2.56] (helo=mail2.digital.com) > by mail.python.org with esmtp (Exim 3.21 #1) > id 14Y2lY-0003Yz-00; Wed, 28 Feb 2001 04:19:25 -0500 > Received: from src-news.pa.dec.com (src-news.pa.dec.com [16.4.16.111]) > by mail2.digital.com (8.9.2/8.9.3/WV2.0h) with ESMTP id BAA25142; > Wed, 28 Feb 2001 01:19:13 -0800 (PST) > Received: by src-news.pa.dec.com; id BAA15660; Wed, 28 Feb 2001 01:17:57 -0800 (PST) > To: dec-mail-lists-python-list@usenet.pa.dec.com > Path: pa.dec.com!owner-python-list-redist > From: jack@oratrix.nl (Jack Jansen) > Newsgroups: dec.mail.lists.python-list > Subject: Re: [Distutils] Re: CPAN functionality for python - requirements > Organization: Systems Research Center, Compaq Computer Corporation > Lines: 16 > Distribution: dec > Message-ID: <20010228091351.C4A41371690@snelboot.oratrix.nl> > To: Doug Hellmann > Cc: Bruce Sass , > Sean Reifschneider , python-list@python.org, > distutils-sig@python.org > In-Reply-To: Message by Doug Hellmann , > Tue, 27 Feb 2001 21:03:43 -0500 , <01022721090603.20069@branagan> > Mime-Version: 1.0 > Content-Type: text/plain; charset=us-ascii > X-List-Id: General discussion list for the Python programming language > X-Beenthere: python-list@python.org > X-Phone: +31 20 6795452(work), +31 20 6795309 (fax), +31 20 6160335(home) > X-List-Subscribe: , > > X-Errors-To: python-list-admin@python.org > X-List-Unsubscribe: , > > X-List-Post: > X-Mini-Review: Sixties punkrock with an ATTITUDE! > X-Mailman-Version: 2.0.1 (101270) > X-Mailer: exmh version 2.1.0 04/14/1999 > X-Received: by src-news.pa.dec.com; id BAA13907; Wed, 28 Feb 2001 01:17:55 -0800 (PST) > X-Received: by pobox1.pa.dec.com (5.65v3.2/1.1.10.5/07Nov97-1157AM) > id AA08971; Wed, 28 Feb 2001 01:17:55 -0800 > X-Received: from zmamail01.nz-tay.cpqcorp.net by pobox1.pa.dec.com (5.65v3.2/1.1.10.5/07Nov97-1157AM) > id AA14595; Wed, 28 Feb 2001 01:17:54 -0800 > X-Received: by zmamail01.zma.compaq.com (Postfix, from userid 12345) > id A497011C86; Wed, 28 Feb 2001 04:17:30 -0500 (EST) > X-List-Archive: > X-Received: from mail.python.org (mail.python.org [63.102.49.29]) > by zmamail01.zma.compaq.com (Postfix) with ESMTP id 7DFCAC1AB > for ; Wed, 28 Feb 2001 04:17:27 -0500 (EST) > X-Received: from localhost.localdomain ([127.0.0.1] helo=mail.python.org) > by mail.python.org with esmtp (Exim 3.21 #1) > id 14Y2hV-0003IL-00; Wed, 28 Feb 2001 04:15:13 -0500 > X-Received: from [212.136.221.130] (helo=oratrix.oratrix.nl) > by mail.python.org with esmtp (Exim 3.21 #1) > id 14Y2gO-00038k-00; Wed, 28 Feb 2001 04:14:04 -0500 > X-Organisation: Oratrix Development, Valeriusplein 30, 1075 BJ Amsterdam, NL > X-Received: from snelboot.oratrix.nl (snelboot.oratrix.nl [212.136.221.131]) > by oratrix.oratrix.nl (Postfix) with ESMTP > id E61C211CA2A; Wed, 28 Feb 2001 10:14:03 +0100 (MET) > X-Precedence: bulk > X-Received: from snelboot.oratrix.nl (localhost [127.0.0.1]) > by snelboot.oratrix.nl (Postfix) with ESMTP > id C4A41371690; Wed, 28 Feb 2001 10:13:51 +0100 (MET) > X-List-Help: > X-Last-Band-Seen: Hives (Melkweg, 25-2) > Sender: distutils-sig-admin@python.org > Errors-To: distutils-sig-admin@python.org > X-BeenThere: distutils-sig@python.org > Precedence: bulk > List-Help: > List-Post: > List-Subscribe: , > > List-Id: Python Distribution Utilities > List-Unsubscribe: , > > List-Archive: > X-Original-Date: 28 Feb 2001 01:17:57 -0800 > Date: 28 Feb 2001 01:17:57 -0800 > > > > I'm not sure why the "what do you have" question is needed. The "send me > > that(mandrake.rpm)" interaction is what we want. > > Well, one of the things I'd like to do is integrate all this in MacPython IDE, > and the same probably holds for PythonWin and Idle. Then the "what do you > have" is pretty useful for building menus and such. > -- > Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ > Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ > www.oratrix.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm > > > > -- > http://mail.python.org/mailman/listinfo/python-list > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG@python.org > http://mail.python.org/mailman/listinfo/distutils-sig From guido@digicool.com Wed Feb 28 10:34:01 2001 From: guido@digicool.com (Guido van Rossum) Date: Wed Feb 28 10:34:01 2001 Subject: [Distutils] Please disable your mail mirror, whoever it is! In-Reply-To: Your message of "28 Feb 2001 01:17:57 PST." <20010228091351.C4A41371690@snelboot.oratrix.nl> References: <20010228091351.C4A41371690@snelboot.oratrix.nl> Message-ID: <200102281526.KAA28444@cj20424-a.reston1.va.home.com> Somebody or something is echoing all mail in this thread. Below is a sample message -- look at all the repeated headers with an X- in front of them. Can this please stop? --Guido van Rossum (home page: http://www.python.org/~guido/) > Return-Path: distutils-sig-admin@python.org > Delivery-Date: Wed Feb 28 04:25:40 2001 > Return-Path: > Received: from localhost (IDENT:guido@localhost.localdomain [127.0.0.1]) > by cj20424-a.reston1.va.home.com (8.9.3/8.9.3) with ESMTP id EAA23220 > for ; Wed, 28 Feb 2001 04:25:40 -0500 > Received: from pop.digicool.com > by localhost with POP3 (fetchmail-5.3.1) > for guido@localhost (single-drop); Wed, 28 Feb 2001 04:25:40 -0500 (EST) > Received: from ns2.digicool.com ([216.164.72.2] verified) > by digicool.com (CommuniGate Pro SMTP 3.4) > with ESMTP id 1628614 for guido@mail.digicool.com; Wed, 28 Feb 2001 04:24:25 -0500 > Received: from mail.python.org (mail.python.org [63.102.49.29]) > by ns2.digicool.com (8.9.3/8.9.3) with ESMTP id EAA06781 > for ; Wed, 28 Feb 2001 04:22:01 -0500 > Received: from localhost.localdomain ([127.0.0.1] helo=mail.python.org) > by mail.python.org with esmtp (Exim 3.21 #1) > id 14Y2mB-0003d5-00; Wed, 28 Feb 2001 04:20:03 -0500 > Received: from [204.123.2.56] (helo=mail2.digital.com) > by mail.python.org with esmtp (Exim 3.21 #1) > id 14Y2lY-0003Yz-00; Wed, 28 Feb 2001 04:19:25 -0500 > Received: from src-news.pa.dec.com (src-news.pa.dec.com [16.4.16.111]) > by mail2.digital.com (8.9.2/8.9.3/WV2.0h) with ESMTP id BAA25142; > Wed, 28 Feb 2001 01:19:13 -0800 (PST) > Received: by src-news.pa.dec.com; id BAA15660; Wed, 28 Feb 2001 01:17:57 -0800 (PST) > To: dec-mail-lists-python-list@usenet.pa.dec.com > Path: pa.dec.com!owner-python-list-redist > From: jack@oratrix.nl (Jack Jansen) > Newsgroups: dec.mail.lists.python-list > Subject: Re: [Distutils] Re: CPAN functionality for python - requirements > Organization: Systems Research Center, Compaq Computer Corporation > Lines: 16 > Distribution: dec > Message-ID: <20010228091351.C4A41371690@snelboot.oratrix.nl> > To: Doug Hellmann > Cc: Bruce Sass , > Sean Reifschneider , python-list@python.org, > distutils-sig@python.org > In-Reply-To: Message by Doug Hellmann , > Tue, 27 Feb 2001 21:03:43 -0500 , <01022721090603.20069@branagan> > Mime-Version: 1.0 > Content-Type: text/plain; charset=us-ascii > X-List-Id: General discussion list for the Python programming language > X-Beenthere: python-list@python.org > X-Phone: +31 20 6795452(work), +31 20 6795309 (fax), +31 20 6160335(home) > X-List-Subscribe: , > > X-Errors-To: python-list-admin@python.org > X-List-Unsubscribe: , > > X-List-Post: > X-Mini-Review: Sixties punkrock with an ATTITUDE! > X-Mailman-Version: 2.0.1 (101270) > X-Mailer: exmh version 2.1.0 04/14/1999 > X-Received: by src-news.pa.dec.com; id BAA13907; Wed, 28 Feb 2001 01:17:55 -0800 (PST) > X-Received: by pobox1.pa.dec.com (5.65v3.2/1.1.10.5/07Nov97-1157AM) > id AA08971; Wed, 28 Feb 2001 01:17:55 -0800 > X-Received: from zmamail01.nz-tay.cpqcorp.net by pobox1.pa.dec.com (5.65v3.2/1.1.10.5/07Nov97-1157AM) > id AA14595; Wed, 28 Feb 2001 01:17:54 -0800 > X-Received: by zmamail01.zma.compaq.com (Postfix, from userid 12345) > id A497011C86; Wed, 28 Feb 2001 04:17:30 -0500 (EST) > X-List-Archive: > X-Received: from mail.python.org (mail.python.org [63.102.49.29]) > by zmamail01.zma.compaq.com (Postfix) with ESMTP id 7DFCAC1AB > for ; Wed, 28 Feb 2001 04:17:27 -0500 (EST) > X-Received: from localhost.localdomain ([127.0.0.1] helo=mail.python.org) > by mail.python.org with esmtp (Exim 3.21 #1) > id 14Y2hV-0003IL-00; Wed, 28 Feb 2001 04:15:13 -0500 > X-Received: from [212.136.221.130] (helo=oratrix.oratrix.nl) > by mail.python.org with esmtp (Exim 3.21 #1) > id 14Y2gO-00038k-00; Wed, 28 Feb 2001 04:14:04 -0500 > X-Organisation: Oratrix Development, Valeriusplein 30, 1075 BJ Amsterdam, NL > X-Received: from snelboot.oratrix.nl (snelboot.oratrix.nl [212.136.221.131]) > by oratrix.oratrix.nl (Postfix) with ESMTP > id E61C211CA2A; Wed, 28 Feb 2001 10:14:03 +0100 (MET) > X-Precedence: bulk > X-Received: from snelboot.oratrix.nl (localhost [127.0.0.1]) > by snelboot.oratrix.nl (Postfix) with ESMTP > id C4A41371690; Wed, 28 Feb 2001 10:13:51 +0100 (MET) > X-List-Help: > X-Last-Band-Seen: Hives (Melkweg, 25-2) > Sender: distutils-sig-admin@python.org > Errors-To: distutils-sig-admin@python.org > X-BeenThere: distutils-sig@python.org > Precedence: bulk > List-Help: > List-Post: > List-Subscribe: , > > List-Id: Python Distribution Utilities > List-Unsubscribe: , > > List-Archive: > X-Original-Date: 28 Feb 2001 01:17:57 -0800 > Date: 28 Feb 2001 01:17:57 -0800 > > > > I'm not sure why the "what do you have" question is needed. The "send me > > that(mandrake.rpm)" interaction is what we want. > > Well, one of the things I'd like to do is integrate all this in MacPython IDE, > and the same probably holds for PythonWin and Idle. Then the "what do you > have" is pretty useful for building menus and such. > -- > Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ > Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ > www.oratrix.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm > > > > -- > http://mail.python.org/mailman/listinfo/python-list > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG@python.org > http://mail.python.org/mailman/listinfo/distutils-sig -- http://mail.python.org/mailman/listinfo/python-list From thomas.heller@ion-tof.com Wed Feb 28 10:47:00 2001 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Wed Feb 28 10:47:00 2001 Subject: [Distutils] Distutils suggestion - test action References: <714DFA46B9BBD0119CD000805FC1F53B01B5AD45@UKRUX002.rundc.uk.origin-it.com> <20010228074901.A17704@newcnri.cnri.reston.va.us> Message-ID: <03a901c0a19d$903eb520$e000a8c0@thomasnotebook> I still have this code in one of my setup-scripts, the original version should be still in the archives, also some discussion about a test command: class test(Command): # Original version of this class posted # by Berthold H=F6llmann to distutils-sig@python.org description =3D "test the distribution prior to install" user_options =3D [ ('test-dir=3D', None, "directory that contains the test definitions"), ('test-prefix=3D', None, "prefix to the testcase filename"), ('test-suffixes=3D', None, "a list of suffixes used to generate names the of the testcases"= ) ] def initialize_options(self): self.build_base =3D 'build' # these are decided only after 'build_base' has its final value # (unless overridden by the user or client) self.test_dir =3D 'test' self.test_prefix =3D 'test_' self.test_suffixes =3D None # initialize_options() def finalize_options(self): import os if self.test_suffixes is None: self.test_suffixes =3D [] pref_len =3D len(self.test_prefix) for file in os.listdir(self.test_dir): if (file[-3:] =3D=3D ".py" and file[:pref_len]=3D=3Dself.test_prefix): self.test_suffixes.append(file[pref_len:-3]) build =3D self.get_finalized_command('build') self.build_purelib =3D build.build_purelib self.build_platlib =3D build.build_platlib # finalize_options() def run(self): import sys # Invoke the 'build' command to "build" pure Python modules # (ie. copy 'em into the build tree) self.run_command('build') # remember old sys.path to restore it afterwards old_path =3D sys.path[:] # extend sys.path sys.path.insert(0, self.build_purelib) sys.path.insert(0, self.build_platlib) sys.path.insert(0, self.test_dir) # build include path for test for case in self.test_suffixes: TEST =3D __import__(self.test_prefix+case, globals(), locals(), ['']) try: tested_modules =3D TEST.tested_modules except AttributeError: tested_modules =3D None else: from code_coverage import Coverage coverage =3D Coverage(modules=3Dtested_modules) sys.settrace(coverage.trace) TEST.test() if tested_modules is not None: # reload tested modules to get coverage of imports, etc. for name in tested_modules: module =3D sys.modules.get(name) if module: reload(module) sys.settrace(None) sys.stdout.write("code coverage:\n") coverage.write_results(sys.stdout) # restore sys.path sys.path =3D old_path[:] # run() # class test From robin@alldunn.com Wed Feb 28 11:52:13 2001 From: robin@alldunn.com (Robin Dunn) Date: Wed Feb 28 11:52:13 2001 Subject: [Distutils] The KISS principle [ Re: CPAN functionality for python - requirements ] References: <714DFA46B9BBD0119CD000805FC1F53B01B5AD44@UKRUX002.rundc.uk.origin-it.com> Message-ID: <08af01c0a1a6$800b2130$0100a8c0@Rogue> > > But writing a "perfect" utility for automated download-and-install, with > dependency tracking, etc etc, is going to be VERY HARD. Don't get misled - > Perl doesn't have such a beast. And We won't even have what Perl has if we > focus on perfection rather than practicality. Agreed. There is an old saying about shooting for the stars to reach the moon. Some of the ideas and proposals for all this seem to be shooting for distant galaxies and are unable to even get off the ground. IMHO we should start with something simple that allows for some of the ultimate goals, and that can be built upon later to get to more of the goals. Here is what I think the simple version needs: 1. A standard place where a file containing a list of mirrors and maybe a bit of mirror meta-data can be fetched from. 2. A python module to parse that mirror data into a meaningful data structure such as a dictionary or a list of instance objects of some type. 3. At every site in the mirrors file there will be a file containing meta-data about all the packages in the archive. 4. A python module to parse the package meta-data into something the client code can easily use. (The meta-data format is probably XML, but hiding it like this means the tool developer doesn't need to know or care what format the file is in. Ditto for the mirror list.) 5. An automated way to extract at least most if not all of the package meta-data from a Distutils based package and upload the meta-data, the sdist, and any bdists that the developer has made to the archive. This could be a new command added to Distutils, with a bit of functionality on the server side for receiving the files, moving them into the "right place" in the archive, and updating the package meta-data file. 6. Define a file format and supporting Python code for tracking which packages and versions have been fetched from the network and installed. Again, this should probably be a function of Distutils so it will also catch the cases where you downloaded some package not in the archive network and ran "python setup.py install" yourself. That's it. With just that bit in place intelligent clients could be written as command-line or GUI tools, or even a web-based interface. They simply fetch the list of mirrors and fetch the package meta-data from a desired mirror and cache this info locally. Then the client tools can do things without having to talk to a server like list available packages, query dependencies, list packages you have installed, etc. When you want to fetch and install a package then the client tool can use the package meta-data and urllib to fetch the sdist and/or a desired bdist and then optionally install it either using the package's setup.py or by whatever command is neccessary for the bdist. -- Robin Dunn Software Craftsman robin@AllDunn.com Java give you jitters? http://wxPython.org Relax with wxPython! From robin@alldunn.com Wed Feb 28 11:57:04 2001 From: robin@alldunn.com (Robin Dunn) Date: Wed Feb 28 11:57:04 2001 Subject: [Distutils] The KISS principle [ Re: CPAN functionality for python - requirements ] References: <714DFA46B9BBD0119CD000805FC1F53B01B5AD44@UKRUX002.rundc.uk.origin-it.com> Message-ID: <08af01c0a1a6$800b2130$0100a8c0@Rogue> > > But writing a "perfect" utility for automated download-and-install, with > dependency tracking, etc etc, is going to be VERY HARD. Don't get misled - > Perl doesn't have such a beast. And We won't even have what Perl has if we > focus on perfection rather than practicality. Agreed. There is an old saying about shooting for the stars to reach the moon. Some of the ideas and proposals for all this seem to be shooting for distant galaxies and are unable to even get off the ground. IMHO we should start with something simple that allows for some of the ultimate goals, and that can be built upon later to get to more of the goals. Here is what I think the simple version needs: 1. A standard place where a file containing a list of mirrors and maybe a bit of mirror meta-data can be fetched from. 2. A python module to parse that mirror data into a meaningful data structure such as a dictionary or a list of instance objects of some type. 3. At every site in the mirrors file there will be a file containing meta-data about all the packages in the archive. 4. A python module to parse the package meta-data into something the client code can easily use. (The meta-data format is probably XML, but hiding it like this means the tool developer doesn't need to know or care what format the file is in. Ditto for the mirror list.) 5. An automated way to extract at least most if not all of the package meta-data from a Distutils based package and upload the meta-data, the sdist, and any bdists that the developer has made to the archive. This could be a new command added to Distutils, with a bit of functionality on the server side for receiving the files, moving them into the "right place" in the archive, and updating the package meta-data file. 6. Define a file format and supporting Python code for tracking which packages and versions have been fetched from the network and installed. Again, this should probably be a function of Distutils so it will also catch the cases where you downloaded some package not in the archive network and ran "python setup.py install" yourself. That's it. With just that bit in place intelligent clients could be written as command-line or GUI tools, or even a web-based interface. They simply fetch the list of mirrors and fetch the package meta-data from a desired mirror and cache this info locally. Then the client tools can do things without having to talk to a server like list available packages, query dependencies, list packages you have installed, etc. When you want to fetch and install a package then the client tool can use the package meta-data and urllib to fetch the sdist and/or a desired bdist and then optionally install it either using the package's setup.py or by whatever command is neccessary for the bdist. -- Robin Dunn Software Craftsman robin@AllDunn.com Java give you jitters? http://wxPython.org Relax with wxPython! -- http://mail.python.org/mailman/listinfo/python-list From jafo@tummy.com Wed Feb 28 18:55:13 2001 From: jafo@tummy.com (Sean Reifschneider) Date: Wed Feb 28 18:55:13 2001 Subject: [Distutils] Re: CPAN functionality for python - requirements In-Reply-To: <20010228091351.C4A41371690@snelboot.oratrix.nl>; from jack@oratrix.nl on Wed, Feb 28, 2001 at 10:13:51AM +0100 References: <20010228091351.C4A41371690@snelboot.oratrix.nl> Message-ID: <20010228165449.A1781@tummy.com> On Wed, Feb 28, 2001 at 10:13:51AM +0100, Jack Jansen wrote: >Well, one of the things I'd like to do is integrate all this in MacPython IDE, >and the same probably holds for PythonWin and Idle. Then the "what do you >have" is pretty useful for building menus and such. I anticipate having a web-based interface to the database as well. I'm sure that we can get the CGI to let the user do that sort of search, but at the moment it's restricted so that the user has to specify the name of the package they're looking for. I figured dumping gigabytes across some user's line wouldn't be optimal. ;-) Sean -- The sooner you start to code, the longer the program will take. -- Roy Carlson Sean Reifschneider, Inimitably Superfluous tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python From jafo@tummy.com Wed Feb 28 19:08:03 2001 From: jafo@tummy.com (Sean Reifschneider) Date: Wed Feb 28 19:08:03 2001 Subject: [Distutils] Re: CPAN functionality for python - requirements In-Reply-To: <714DFA46B9BBD0119CD000805FC1F53B01B5AD44@UKRUX002.rundc.uk.origin-it.com>; from Paul.Moore@uk.origin-it.com on Wed, Feb 28, 2001 at 09:59:38AM -0000 References: <714DFA46B9BBD0119CD000805FC1F53B01B5AD44@UKRUX002.rundc.uk.origin-it.com> Message-ID: <20010228170725.B1781@tummy.com> On Wed, Feb 28, 2001 at 09:59:38AM -0000, Moore, Paul wrote: >MakeMaker >--------- Handled by distutils, as you mention. >The system of FTP sites and mirrors >----------------------------------- Depends on wether you want a single central location, or the ability for packages to reside on a disperse set of machines... My setup can handle either. >CPAN.pm >------- This is what I'm working on, though it's interface to the catalog is fairly stand-alone instead of being dumped in a bunch of FTP directories (as I understand being the way CPAN works). >enough that things like FTP client support is available out of the box >(whereas CPAN.pm won't work until you manually install libnet, and possibly Well, yes and no... Unfortunately, using urlretrieve on an ftp:// URL seems to be broken at best. In my prototyping work last night, I ended up just calling "lftp" via os.system() -- not optimal but it was at least working. I need to dig into urllib some more, but at the least it seemed to need to run in passive mode, have better handling of malformed URLs (ftp://ftp.tummy.com//foo/bar/ was causing problems because of the doubled "/" created by appending a base and mirror URL). There also seemed to be a problem with closing the session properly when talking to an anonfile server... >But writing a "perfect" utility for automated download-and-install, with >dependency tracking, etc etc, is going to be VERY HARD. Don't get misled - Can't be that hard. I'm still working on the dependencies, but I've been able to get the rest of it working in my copious spare time before going to bed over the last two nights... >The h2xs program >---------------- That seems like a distutils tool to me. >A social structure >------------------ Based on my experimenting with Distutils last night, I don't know that this will be a problem. Distutils is totally cool -- I can't imagine going back. >Interestingly, developers probably don't "expect" to have to include >dependency information, and hence many don't - resulting in the problems you I think we can deal with that in an iterative manner. First get them to build distutils packages, then when it fails to install on some user's machine because they don't have foo.py we can educate them on the joys of listing third-party module requirements. >still (relatively) new, and so there are LOTS of packages which don't come >with a setup.py yet. Often, adding one isn't hard, but it is still to The biggest win of using distutils is that it makes it easier for the developer to run their software on multiple machines. That selfish reason is enough for me. >Things are looking better with Python 2.1, though. Included with 2.1, it >looks like there will be a standard unit testing module, and a new "pydoc" >package, which will extract documentation from module docstrings. So the Should be sweet... Sean -- That weapon will replace your tongue. You will learn to speak through it. And your poetry will now be written with blood. -- _Dead_Man_ Sean Reifschneider, Inimitably Superfluous tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python