From jpe@archaeopteryx.com Fri Oct 6 20:48:19 2000 From: jpe@archaeopteryx.com (John Ehresman) Date: Fri, 6 Oct 2000 15:48:19 -0400 (EDT) Subject: [Compiler-sig] New version of parsetools Message-ID: Hi, I've put a new version of our parsetools package on our ftp server at ftp://archaeopteryx.com/pub/parsetools/parsetools-0.9.2.tgz The package contains a tokenizer for either Python 1.5.2 or Python 2.0 source, an interface to the builtin 'pgen' parser that bypasses Python's tokenizer, and a parse tree to AST transformer which works with parse trees from the 1.5.2 & 2.0 parsers. The major changes from the previous version is support for Python 2.0, the ability to compile a grammar into the pgen module so it's grammar can differ from the grammar in the core, and the storage of positions of individual tokens in the source. Included is a transformer_test.py script which will parse & transform each line of given source files to an AST. It's also the beginnings of Python syntax checker with line-by-line error recovery; at most one syntax error will be found on every line. Running it over a C or sh file results in a error for nearly every line (except where the source happens to be valid Python). It's not complete, though, because indentation is ignored and multi-line constructs, such as if ... :\n ...\n else:\n..., are not checked for correctness. Please let me know of any problems, John ------------------------------------------------------------------------ Archaeopteryx Software, Inc. Wing IDE for Python www.archaeopteryx.com Take Flight! From MarkH@ActiveState.com Tue Oct 10 13:14:32 2000 From: MarkH@ActiveState.com (Mark Hammond) Date: Tue, 10 Oct 2000 23:14:32 +1100 Subject: [Compiler-sig] Update to compiler for Python 2.0 Message-ID: Hi all, The following patch allows the nondist/src/Compiler package to pass its test suite again. Notes re transformer: * The code was changed in such a way it will still work with Python 1.x. Thus, I avoid using new 2.0 specific "symbol" module attributes. The other files are changed in a 2.0 specific way. * The import changes seem hacky, but it appears the correct solution would be to introduce a new node just for the import name. Couldn't be bothered! * There are no changes to implement "import as", but the current test suite passes again... If there are no objections over the next few days, I will check it in... Mark. Index: compiler/pyassem.py =================================================================== RCS file: /projects/cvsroot/python/nondist/src/Compiler/compiler/pyassem.py,v retrieving revision 1.9 diff -r1.9 pyassem.py 516c516 < # UNPACK_TUPLE, UNPACK_LIST, BUILD_TUPLE, --- > # UNPACK_SEQUENCE, BUILD_TUPLE, 518,520c518 < def UNPACK_TUPLE(self, count): < return count < def UNPACK_LIST(self, count): --- > def UNPACK_SEQUENCE(self, count): Index: compiler/pycodegen.py =================================================================== RCS file: /projects/cvsroot/python/nondist/src/Compiler/compiler/pycodegen.py,v retrieving revision 1.20 diff -r1.20 pycodegen.py 459c459 < self.emit('UNPACK_TUPLE', len(node.nodes)) --- > self.emit('UNPACK_SEQUENCE', len(node.nodes)) 709c709 < self.emit('UNPACK_TUPLE', len(tup)) --- > self.emit('UNPACK_SEQUENCE', len(tup)) Index: compiler/transformer.py =================================================================== RCS file: /projects/cvsroot/python/nondist/src/Compiler/compiler/transformer.py,v retrieving revision 1.10 diff -r1.10 transformer.py 403c403,413 < names.append(nodelist[i][1]) --- > this_node = nodelist[i] > if type(this_node[1]) == type(''): > # Old 1.6 style > this_name = this_node[1] > else: > # import foo as spam > this_name = this_node[1][1] > if len(this_node) > 2: > as_name = this_node[3][1] > this_name = this_name, as_name > names.append(this_name) 409c419,430 < names.append(self.com_dotted_name(nodelist[i])) --- > this_node = nodelist[i] > if this_node[0] == symbol.dotted_name: > # Old 1.6 style > this_name = self.com_dotted_name(this_node) > else: # symbol.dotted_as_name, but we can't use that in 1.6 > # import foo as spam > this_name = self.com_dotted_name(this_node[1]) > if len(this_node) > 2: > as_name = this_node[3][1] > this_name = this_name, as_name > names.append(this_name) > From jeremy@beopen.com Thu Oct 12 00:40:11 2000 From: jeremy@beopen.com (Jeremy Hylton) Date: Wed, 11 Oct 2000 19:40:11 -0400 (EDT) Subject: [Compiler-sig] Update to compiler for Python 2.0 In-Reply-To: References: Message-ID: <14820.64091.389620.994877@bitdiddle.concentric.net> Mark, One question that Neil Schemenauer raised (privately I think) is whether the compiler package in 2.0 should be compatible with earlier versions of Python. I see that the changes you have made will allow the code to run with Python 1.5.2. How important is this? I am planning to implement the new features introduced in 2.0, like augmented assignment and list comprehensions. It's not clear that I can do this without introducing a dependency on 2.0. If it is important to have pre-2.0 compatibility, how should we maintain the code? I imagine either a separate SF project with just that code or a set of subclasses that can be used with old versions of Python. Do either of these make sense? Jeremy From MarkH@ActiveState.com Thu Oct 12 00:48:24 2000 From: MarkH@ActiveState.com (Mark Hammond) Date: Thu, 12 Oct 2000 10:48:24 +1100 Subject: [Compiler-sig] Update to compiler for Python 2.0 In-Reply-To: <14820.64091.389620.994877@bitdiddle.concentric.net> Message-ID: > One question that Neil Schemenauer raised (privately I think) is > whether the compiler package in 2.0 should be compatible with earlier > versions of Python. I see that the changes you have made will allow > the code to run with Python 1.5.2. How important is this? ATM, I am using transformer.py and related code for the .NET project. I don't want to insist on Python 2.0 yet for this, but will be happy to in a few months. I don't care about the compiler and assembler. > If it is important to have pre-2.0 compatibility, how should we > maintain the code? I imagine either a separate SF project with just > that code or a set of subclasses that can be used with old versions of > Python. Do either of these make sense? It is is a problem in the short-term, I simply won't synch my changes with yours... Mark. From kmullin@uswest.com Fri Oct 13 19:47:54 2000 From: kmullin@uswest.com (Kevin Mullin) Date: Fri, 13 Oct 2000 11:47:54 -0700 Subject: [Compiler-sig] Problem installing Python1.4 on OS/390 V2R10 Message-ID: <39E758DA.B64AA7F4@uswest.com> I hope someone out there has already tried this and can help me. There is an IBM website at http://www.s390.ibm.com/products/oe/python.html with instructions on installing Python 1.4 on OS/390. I've followed those instructions to install Python 1.4 (hopefully, later I'll go to 2.0). After following the instructions on this site, I do the configure (which completes fine) and the make. But the make generates the following: cc -O -o python \ IEW2785S D21A AN ATTEMPT TO OBTAIN FILE STATISTICS FOR PATHNAME ./ FAILED. .HFS. ISSUED RETURN CODE 00000081 AND REASON CODE 053B006C. IEW2470E 9511 ORDERED SECTION CEESTART NOT FOUND IN MODULE. IEW2648E 5111 ENTRY CEESTART IS NOT A CSECT OR AN EXTERNAL NAME IN THE MODULE. FSUM3065 The LINKEDIT step ended with return code 12. Has anyone ever seen this? Since it seems to be complaining about ./, I changed every directory in the path of Python1.4 (which resulted from the pax) to 777. Also, Return Code 00000081 is documented in the USS Messages and Codes as saying simply that the file/directory is not found. Unfortunately, the Reason Code is not documented. Help! -- Kevin L. Mullin Qwest Communications MVS Support Phone: (425) 451-6045 Fax: (425) 453-3589 Email: kmullin@uswest.com From jeremy@beopen.com Fri Oct 13 23:09:39 2000 From: jeremy@beopen.com (Jeremy Hylton) Date: Fri, 13 Oct 2000 18:09:39 -0400 (EDT) Subject: [Compiler-sig] Update to compiler for Python 2.0 In-Reply-To: References: <14820.64091.389620.994877@bitdiddle.concentric.net> Message-ID: <14823.34851.10396.311706@bitdiddle.concentric.net> I ended up taking a brute force approach to this problem: I implemented the missing bits for 2.0 and then added backwards compatibilty support for 1.5.2. I tested this using the new regrtest script, which compiles and runs the entire test suite to make sure that the compiler works. I hope this doesn't interfere with any of the .NET stuff you are doing. If there are any problems, I'll try to fix them. Jeremy From jeremy@beopen.com Sat Oct 14 00:17:54 2000 From: jeremy@beopen.com (Jeremy Hylton) Date: Fri, 13 Oct 2000 19:17:54 -0400 (EDT) Subject: [Compiler-sig] [ANNOUNCE] Ninth International Python Conference Message-ID: <14823.38946.438300.812362@bitdiddle.concentric.net> CALL FOR PAPERS, POSTERS, AND PARTICIPATION Ninth International Python Conference March 5-8, 2001 Long Beach, California Web site: http://python9.org The 9th International Python Conference (Python 9) is a forum for Python users, software developers, and researchers to present current work, discuss future plans for the language and commercial applications, and learn about interesting uses of Python. The conference will consist of a day of tutorials, two days of refereed paper and application tracks -- including a Zope track and a multi-technology Python Applications Track, a developers' day, a small exhibition, demonstrations, and posters. Paper submission deadline: Monday, November 6, 2000 Notification of acceptance: Monday, December 11, 2000 Final papers due: Monday, January 15, 2001 Authors are invited to submit papers for the Refereed Paper Track that: * Present new and useful applications and tools that utilize Python * Describe the use of Python in large, mission-critical or unusual applications * Address practical programming problems, and provide lessons based on experience for Python programmers Full information is available on the website, http://python9.org From pegmgr@emily.peg.com Mon Oct 30 18:57:53 2000 From: pegmgr@emily.peg.com (PEG Manager) Date: Mon, 30 Oct 2000 12:57:53 -0600 Subject: [Compiler-sig] Compiler Error Under BSDI 4.1 Message-ID: <200010301857.MAA05227@emily.peg.com> Does anyone have a quick and dirty fix to let me get 2.0 compiled. My c skills are way too rusty to attack this in the limited time I have available. From Objects/fileobject.c ... /* define the appropriate 64-bit capable tell() function */ #if defined(MS_WIN64) #define TELL64 _telli64 #elif defined(__NetBSD__) || defined(__OpenBSD__) /* NOTE: this is only used on older NetBSD prior to f*o() funcions */ #define TELL64(fd) lseek((fd),0,SEEK_CUR) #endif ... /* a portable fseek() function return 0 on success, non-zero on failure (with errno set) */ int #if defined(HAVE_LARGEFILE_SUPPORT) && SIZEOF_OFF_T < 8 && SIZEOF_FPOS_T >= 8 _portable_fseek(FILE *fp, fpos_t offset, int whence) #else _portable_fseek(FILE *fp, off_t offset, int whence) #endif { #if defined(HAVE_FSEEKO) return fseeko(fp, offset, whence); #elif defined(HAVE_FSEEK64) return fseek64(fp, offset, whence); #elif defined(__BEOS__) return _fseek(fp, offset, whence); #elif defined(HAVE_LARGEFILE_SUPPORT) && SIZEOF_FPOS_T >= 8 /* lacking a 64-bit capable fseek() (as Win64 does) use a 64-bit capable fsetpos() and tell() to implement fseek()*/ fpos_t pos; switch (whence) { case SEEK_CUR: if (fgetpos(fp, &pos) != 0) return -1; offset += pos; break; case SEEK_END: /* do a "no-op" seek first to sync the buffering so that the low-level tell() can be used correctly */ if (fseek(fp, 0, SEEK_END) != 0) return -1; if ((pos = TELL64(fileno(fp))) == -1L) return -1; offset += pos; break; /* case SEEK_SET: break; */ } return fsetpos(fp, &offset); #else return fseek(fp, offset, whence); #endif } .... During the make: ranlib libpython2.0.a true cd Modules; make OPT="-g -O2 -Wall -Wstrict-prototypes" VERSION="2.0" prefix=" /usr/local" exec_prefix="/usr/local" LIBRARY=../libpython2.0.a link gcc -D_HAVE_BSDI -Xlinker -export-dynamic python.o ../libpython2.0.a -ldl - lutil -lm -o python ../libpython2.0.a(fileobject.o): In function `_portable_fseek': /home/pegmgr/software/Python-2.0/Objects/fileobject.c:274: undefined reference t o `TELL64' *** Error code 1 Stop. *** Error code 1 Stop. -- PEG Manager pegmgr@peg.com From guido@python.org Mon Oct 30 18:56:25 2000 From: guido@python.org (Guido van Rossum) Date: Mon, 30 Oct 2000 13:56:25 -0500 Subject: [Compiler-sig] Compiler Error Under BSDI 4.1 In-Reply-To: Your message of "Mon, 30 Oct 2000 12:57:53 CST." <200010301857.MAA05227@emily.peg.com> References: <200010301857.MAA05227@emily.peg.com> Message-ID: <200010301856.NAA24895@cj20424-a.reston1.va.home.com> Wrong list, but I'll reply anyway. :-) > Does anyone have a quick and dirty fix to let me get 2.0 compiled. > > My c skills are way too rusty to attack this in the limited time I > have available. > > >From Objects/fileobject.c > > ... > > /* define the appropriate 64-bit capable tell() function */ > #if defined(MS_WIN64) > #define TELL64 _telli64 > #elif defined(__NetBSD__) || defined(__OpenBSD__) > /* NOTE: this is only used on older > NetBSD prior to f*o() funcions */ > #define TELL64(fd) lseek((fd),0,SEEK_CUR) > #endif Add || defined(_HAVE_BSDI) to the #elif line, making it: #elif defined(__NetBSD__) || defined(__OpenBSD__) || defined(_HAVE_BSDI) --Guido van Rossum (home page: http://www.python.org/~guido/) From fredrik@effbot.org Mon Oct 30 19:14:41 2000 From: fredrik@effbot.org (Fredrik Lundh) Date: Mon, 30 Oct 2000 20:14:41 +0100 Subject: [Compiler-sig] Compiler Error Under BSDI 4.1 References: <200010301857.MAA05227@emily.peg.com> Message-ID: <000701c042a5$ab5c16b0$3c6340d5@hagrid> > Does anyone have a quick and dirty fix to let me get 2.0 compiled. wrong mailing list: the compiler-sig is about developing compilers and type analysis systems for python, not about compiling it... > My c skills are way too rusty to attack this in the limited time I > have available. you'll find instructions here: http://www.deja.com/=dnc/getdoc.xp?AN=683926459&fmt=text summary: You can fix this by editing './config.h' after running './configure', and commenting out the HAVE_LARGEFILE_SUPPORT line; changing this: #define HAVE_LARGEFILE_SUPPORT 1 into something like this: /* #define HAVE_LARGEFILE_SUPPORT 1 */ (You can just remove the line as well.) see the post for more info. From hei@adtranzsig.de Tue Oct 31 08:55:01 2000 From: hei@adtranzsig.de (Dirk-Ulrich Heise) Date: Tue, 31 Oct 2000 09:55:01 +0100 Subject: [Compiler-sig] Producing tokens for a Forth machine? References: <200010301857.MAA05227@emily.peg.com> <000701c042a5$ab5c16b0$3c6340d5@hagrid> Message-ID: <00cd01c04318$413a26a0$13ec1fc2@adtranzsig.de> Hi list! I finished the foundation for Forth interpreted in Python. It can be thought of as a Forth with Python objects on the stack, and Python object semantics hold. The Forth machine model is the simplest interpreter i can think of. That's why i want to explore something like this: 1.) Parse Python 2.) produce Forth definitions / tokens for my Forth machine The Forth can call any Python functionality. So, if there was a parser for Python written as a PYthon module, 2.) could be doable for me. Is there a suitable module? [Why would anyone want to do something as crazy as this? 1.) It's easy to port the Forth machine to C/C++ later 2.) I think, things like a peephole optimizer or a type inferencer can take advantage of the simple Forth code model; maybe helping each other.] [The Forth machine is at http://www.egroups.com/group/synthetoforth ] Dipl.Inform. Dirk-Ulrich Heise hei@adtranzsig.de dheise@debitel.net