From gward@cnri.reston.va.us Wed Nov 25 22:47:03 1998 From: gward@cnri.reston.va.us (Greg Ward) Date: Wed, 25 Nov 1998 17:47:03 -0500 (EST) Subject: [Distutils-sig] Test message Message-ID: <199811252247.RAA14749@thrak.cnri.reston.va.us> Hello to everyone out there in distutils-sig land (just me, so far). Let the fun begin! Greg From gward@cnri.reston.va.us Wed Nov 25 22:59:59 1998 From: gward@cnri.reston.va.us (Greg Ward) Date: Wed, 25 Nov 1998 17:59:59 -0500 (EST) Subject: [Distutils-sig] Charter for this SIG Message-ID: <199811252259.RAA14761@thrak.cnri.reston.va.us> distutils-sig: forum to discuss module distribution utilities for Python The distutils sig exists to discuss the design and implementation of a suite of module distribution utilities for Python. These utilities will take the form of some standard modules, probably grouped in the 'distutils' package, for building, packaging, and installing third-party modules. This includes both modules written purely in Python and extension modules written in C/C++. The main deliverable will be the following core modules (in the distutils package): build - provides code for building a module, which might include copying .py files into a staging area, compiling and linking .c files, processing documentation to an installable form, etc. dist - create a source distribution bdist - create a 'built distribution' (the equivalent of a 'binary distribution', except that binaries won't necessarily be present install - install a built library on the local machine gen_make - generate a Makefile to do some of the above tasks (mainly 'build', for developer convenience and efficiency) I will tentatively put forward March 1999 as a target for a working release to run on top of Python 1.5.2, which *might* require a patch-and-rebuild-Python step (or might find ways to work with the information available under 1.5). For a complete, tested, documented suite ready to bundle with Python 1.6, let's shoot for June 1999. Other topics of interest: * encouraging module developers to write test suites by having a standard place for them in module distributions * ditto for documentation -- although this is probably the job of the doc-sig, it would be nice to tie the two together at some point * a standard for representing and comparing version numbers * social engineering in general, ie. convincing module developers to start using the system * the possible need for tweaks to the configure/build process for Python itself, and a place to hold configuration information (possibly a new built-in module, 'sys.config') * possibly rewriting the configure/build/install process for Python 1.6 -- especially useful if the distutils are bundled with 1.6! * ties to Trove -- any module distribution scheme must include a way to describe the package metadata, and there will need to be hooks between any future Trove archive for Python and this metadata * recognizing SWIG-assisted extensions in addition to "ordinary" C extensions The starting point for discussion on the sig will be the summary of the IPC7 Developer's Day session which got this whole thing rolling: http://www.foretec.com/python/workshops/1998-11/dd-ward-sum.html From fleck@informatik.uni-bonn.de Sun Nov 29 01:25:28 1998 From: fleck@informatik.uni-bonn.de (Markus Fleck) Date: Sun, 29 Nov 1998 02:25:28 +0100 (MET) Subject: [Distutils-sig] (fwd) PROPOSAL: XML based Make for C++, Perl, Python, Java Message-ID: <199811290125.CAA13718@sokrates.informatik.uni-bonn.de> From c.l.py: -- forwarded message -- From: Pluto Newsgroups: comp.lang.perl.misc,comp.lang.java.programmer,comp.lang.python,comp.lang.c++ Subject: PROPOSAL: XML based Make for C++, Perl, Python, Java Date: Fri, 27 Nov 1998 11:45:34 -0800 Message-ID: <365F015D.4B86DE60@nospam.com> NNTP-Posting-Host: 10.143.0.235 Perl, Python,etc are scripting languages that I use and that make me very productive relative to the systems level languages like Java and C++ (that I also use). There is a lot of innovation in both scripting and system languages. But Make stays the same - and cryptic as ever. So, I was hoping to challenge the more knowledgeable people on these newsgroups (Perl, Python, Java, C++) to cooperate and bring Make into the 90s. I was hoping to use an XML grammar to describe the Makefile. The goals for this project: 1) Increase productivity by atleast 200%, while reducing typical makefile lines of code by 50%. 2) Use a well defined XML standard. 3) Very human-readable. 4) X-Platform (Must run on Linux, NT, Solaris, AIX, HP-UX, others) 5) X-Language (Must work for C++, Perl, Python, Java, others) Perl's MakeMaker is a wonderful tool. And Perl is very popular (I like Python too!). So, I was hoping to get the Perl community to lead us to the promised land by redoing MakeMaker using the above requirements. And I also hope that Python, Java and C++ will aggresively build on this. For details on Perl's MakeMaker: http://www.perl.com/CPAN/doc/manual/html/lib/ExtUtils/MakeMaker.html ==================================== luckily, while doing a dejanews search, I came across a very interesting discussion on "perl.porters-gw". See the post below by mike808@mo.net: Some::Package 1.23 Other::Module 4.56 /the/first/module/file /second/module/file module/file any other data Some ideas/projects immediately spring to mind: a MakeMaker package definition DTD? a converter from current MakeMaker format to the DTD XML format? a MakeMaker modification to understand the new DTD? an autodetect in MakeMaker to determine the makefile.PL format and parse appropriately. Create Data::Dumper::XML? ==================================== Are the Perl MakeMaker team ready to lead? -- end of forwarded message -- -- /////////////////////////////////////////////////////////////////////////// Markus B Fleck - Student - University of Bonn - CS Dept IV - UNIX support "May you live in interesting times." (ancient Chinese curse) \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\ From skaller@maxtal.com.au Mon Nov 30 23:22:13 1998 From: skaller@maxtal.com.au (John Skaller) Date: Tue, 01 Dec 1998 09:22:13 +1000 Subject: [Distutils-sig] Compiler module available for comment Message-ID: <1.5.4.32.19981130232213.00f4b3e8@triode.net.au> This is to advise that I have already implemented Python modules with the ability to compile C or C++ Python dynamically loadable modules C or C++ external applications and these are incorporated in interscript in the 'compilers' subdirectory. The implementation is not important. What is important is the _interface_ provided. Please examine. I would appreciate any comments on the generality of the interface. In my opinion, these modules should be in the standard distribution, and be hooked by 'import' so that C/C++ extension developers don't have to provide _any_ makefiles or other installation crud. Just copy the C file into the right place and it gets compiled automatically, just like a .py file gets compiled to a .pyc file automatically. Obviously, this technology cannot handle linking to arbitrary libraries in a platform independent way -- but it will handle a large class of C modules of the kind 'Could be done in Python, but C is faster'. ------------------------------------------------------- John Skaller email: skaller@maxtal.com.au http://www.maxtal.com.au/~skaller phone: 61-2-96600850 snail: 10/1 Toxteth Rd, Glebe NSW 2037, Australia From skaller@maxtal.com.au Mon Nov 30 23:22:15 1998 From: skaller@maxtal.com.au (John Skaller) Date: Tue, 01 Dec 1998 09:22:15 +1000 Subject: [Distutils-sig] distutils charter and interscript Message-ID: <1.5.4.32.19981130232215.00f4a578@triode.net.au> From the charter: >These utilities >will take the form of some standard modules, probably grouped in the >'distutils' package, for building, packaging, and installing third-party >modules. This includes both modules written purely in Python and >extension modules written in C/C++. Interscript is designed to do all this. Except, it has a much wider scope: it isn't limited to building Python, and it includes testing and documentation. (And a few other things -- some of which are implemented already, and some of which are not [such as version control]) There's a generic description of requirements in the documentation at http://www.triode.net.au/~skaller/interscript From the topic list: > dist - create a source distribution Almost done: interscript creates a source tree in the doco. It doesn't yet provide the ability to 'tar' up those files, but that is fairly trivial. > bdist - create a 'built distribution' (the equivalent of a 'binary > distribution', except that binaries won't necessarily > be present As above, except that this time, a list of generated files is available. Again, there is not yet a tool for taring up those files. > install - install a built library on the local machine This is MUCH harder. If you read my documentation, and examine the sections on installation control (site-frame, platform-frame, user-frame) you will see I have made fairly advanced provision for installation control. I'm not using any of this yet. My system discriminates the status of the package, and may install it in different places depending on the status. For example, when I download a new version of a package, I might put it into a test directory and test it, before I install it into a more widely accessible place. Furthermore, the install point is conditioned by the authors evaluation: is it alpha, beta, or production software? > gen_make - generate a Makefile to do some of the above tasks > (mainly 'build', for developer convenience and efficiency) Please don't! Please generate PYTHON script if it is necessary. Python runs on all platforms, and it has -- or should have -- access to all the configuration information. If this is not the case, we need to petition Guido to make that information available, or to provide hooks that allow the client to provide that information ONCE on installing python. Make is extremely unreliable: it isn't available on all platforms, it doesn't work properly on all platforms, it is almost impossible to debug unless you are a Make programmer. We need to get as far aware from this as possible. We have something MUCH better already. Python. Lets use it. >I will tentatively put forward March 1999 as a target for a working >release to run on top of Python 1.5.2, which *might* require a >patch-and-rebuild-Python step (or might find ways to work with the >information available under 1.5). For a complete, tested, documented >suite ready to bundle with Python 1.6, let's shoot for June 1999. I don't believe a patch is necessary: what will almost certainly be necessary, however, is that there be some kind of support module that provides relevant information. And that module will have to be written according to specifications, BY THE CLIENT. Of course, standard defaults can and should be provided :-) For example, the compilers module I provide happens to use gnu gcc/g++. So it won't work on NT. Or a Solaris system where the client wants to use the native cc. Etc etc. >Other topics of interest: > * encouraging module developers to write test suites by having > a standard place for them in module distributions Yes. Although interscript allows a slightly different approach: test code is embedded directly in the source and is executed during the build process. The test code isn't installed anywhere; it's written to a temporary file for testing and then deleted afterwards. This technique is not contrary to the idea of a stand-alone test suite. BTW: 'standard place' needs to be more flexible that a fixed subdirectory! > * ditto for documentation -- although this is probably the job of > the doc-sig, it would be nice to tie the two together at some point Again, I agree. Interscript, as a literate programming tool, emphasises documentation. And again, 'standard place' needs to be more flexible than just a fixed subdirectory. [For example, if you have HTML doco, and want networked clients of the software to see it, it may have to be put into the httpd root: inside the actual package would not work] > * a standard for representing and comparing version numbers I think this is very hard. It isn't enough to just use numbers. RCS/CVS provides a tree. With labelled cut points. AFAIK, it cannot handle the important thing: dependencies. In other words, it isn't enough to just have a standard for version identification, although that is a good start. It is also necessary to determine what depends on what. RedHat RPM tries to do that. It is fairly sophisticated, I'm told, but it isn't very reliable: it doesn't know about non-rpm installation, and it has stuffed up my Linux box badly. > * social engineering in general, ie. convincing module developers > to start using the system Hmm :-) I've already implementd half what this SIG's charter wants. Now how can I convince you to use what I've done instead of re-inventing the wheel? > * the possible need for tweaks to the configure/build process for > Python itself, and a place to hold configuration > information (possibly a new built-in module, 'sys.config') Yes. I believe the build needs to be split into two parts: 1) A nasty, hackers build of the bootstrap version 2) a clean python based build of the rest of the system At present, the whole installation is in category (1): and it isn't necessary. (1) cannot be dispensed with. But it can be reduced for an initial installation, and reduced even more for an upgrade. I think this target should be kept in mind, but otherwise deferred for a while: if a suitably powerful and flexible system is developed, Guido will probably use it. But this cannot happen until the system is tested, distributed as standard, and been thoroughly ingrained in the building of third party extensions. [IMHO: Guido mught surprise me:-] That's because it is likely to be 'too risky' before that time: it isn't so simple because the installation system has to be built correctly _first_ before it can be used to build the rest of Python. And it is likely that configuring the installation system is harder than configuring Python. ------------------------------------------------------- John Skaller email: skaller@maxtal.com.au http://www.maxtal.com.au/~skaller phone: 61-2-96600850 snail: 10/1 Toxteth Rd, Glebe NSW 2037, Australia