From skaller@maxtal.com.au Sat Apr 1 01:59:28 2000 From: skaller@maxtal.com.au (John Max Skaller) Date: Sat, 01 Apr 2000 11:59:28 +1000 Subject: [Compiler-sig] __getattr__ inflexibility References: Message-ID: <38E55800.B202C450@maxtal.com.au> Ludvig Svenonius wrote: > > I was wondering about the __getattr__-built-in method. Currently it is > called only if the attribute could not be found in the instance dictionary. > Would it not be more flexible to -always- call it upon referencing an > attribute This won't work because _inside_ the getattr method, we cannot refer to any of the object's attributes, since doing so will recursively invoke the __getattr__ method. This would _also_ destroy the __setattr__ case, since it _relies_ on being able to get the __dict__ attribute of an instance, in order to store attributes (since __setattr__ IS always invoked on set operation): you have to write self.__dict__['attr'] = value in a __setattr__ method to store a value. -- John (Max) Skaller, mailto:skaller@maxtal.com.au 10/1 Toxteth Rd Glebe NSW 2037 Australia voice: 61-2-9660-0850 checkout Vyper http://Vyper.sourceforge.net download Interscript http://Interscript.sourceforge.net From dominio@hem.passagen.se Tue Apr 4 12:11:56 2000 From: dominio@hem.passagen.se (Ludvig Svenonius) Date: Tue, 04 Apr 2000 13:11:56 +0200 Subject: [Compiler-sig] Polymorphism/encapsulation in Python Message-ID: <38E9CDFB.1DBE2400@hem.passagen.se> Many people have been singing their praises to the extreme level of polymorphism found in Python. Having recently migrated from Java to Python, however, I find myself wanting in the area. The main problem may still be the lack of focus on encapsulation, however (and by encapsulation I do not mean baby-sitting the programmer with access modifiers like C++'s 'private'). Even as a Java programmer, I can certainly praise Python's support for multiple inheritance. Java's all the less for not having it. However, Java's facility for interfaces is a pure blessing, and I lack a counterpart for it in Python. As an example, I will bring up recent scenarios that I've been confronted with when working with XML and the DOM using Java and Python. For Java there exists dozens of libraries with support for both core and peripheral XML standards like Namespaces, XPointer, XSLT, etc. The advantage Java has over Python in the area is not so much the availability of existing libraries, but their general applicability. W3C defined their DOM standard for Java in the form of Java interfaces. These interfaces became the standardized foundation for libraries that depend on the DOM in Java. Because of this fact, that all (well, almost) available DOM-based libraries for Java work against the same clearly defined interface sets, as defined by W3C, it is trivial to write a new DOM implementation and with ease provide it with the features it needs to properly interact with previosly written libraried, based on the standard DOM interface. Unfortunately, doing the very same thing in Python is a hassle. There is no clearly defined DOM interface in Python. Of course the standard DOM interface as proposed by the W3C could be used, as it is in implementations like 4DOM. The problem arises when a user needs to write his/her own implementation of the DOM (or anything else that could have a standardized interface) and needs to make this compatible with existing libraries written for Python. To make my DOM compatible with 4DOM, perhaps I need to subclass the 4DOM classes, so I can use 4XPath and 4XSLT with my DOM implementation, but to get my DOM implementation to work with other libraries as well, other measures, depending on the implementation of the library (which is something that according to the OOP principle of encapsulation should be completely irrelevant for me as a user of the library) will likely be necessary. Java's interfaces were designed as a replacement for multiple inheritance. The lack of support for multiple inheritance sometimes makes design in Java grossly unintuitive, however. As I see it, interfaces can, with great benefit, be combinated with multiple inheritance. In such a design multiple inheritance would mainly be a convenience feature, whereas polymorphism would supply the main support for encapsulation/polymorphism with benefit like the ones found in Java. Interfaces in Python could perhaps take the form of a facility which enables the user to validate an instance against a pre-defined list of functions that are expected to be found in the instance, raising an exception in case of a mismatch. This would of course be possible to do manually in any Python function, but that completely misses the point, as the checking is then part of the function implementation, which the user should not have to concern him-/herself with. The interface facility would furthermore be quite useless unless standardized, so that libraries that make use of the interface facility can start to crop up. What I would suggest is looking into providing native support for interfaces in Python to sidestep these problems. The current lack of it seriously takes the edge out of available APIs and libraries written in Python. Perhaps it could be accomodated in Python 1.7 or Python 3K. I know for one thing that I'll never feel comfortable with leaving Java fully behind me until it is in. -- Ludvig Svenonius Researcher, Excosoft AB +46 (8) 752 74 29 ludvig.svenonius@excosoft.se / dominio@hem.passagen.se From skaller@maxtal.com.au Tue Apr 4 14:20:04 2000 From: skaller@maxtal.com.au (John Max Skaller) Date: Tue, 04 Apr 2000 23:20:04 +1000 Subject: [Compiler-sig] Polymorphism/encapsulation in Python References: <38E9CDFB.1DBE2400@hem.passagen.se> Message-ID: <38E9EC04.B936EB90@maxtal.com.au> Ludvig Svenonius wrote: > > Many people have been singing their praises to the extreme level of > polymorphism found in Python. Having recently migrated from Java to > Python, however, I find myself wanting in the area. The main problem may > still be the lack of focus on encapsulation, however (and by > encapsulation I do not mean baby-sitting the programmer with access > modifiers like C++'s 'private'). > > Even as a Java programmer, I can certainly praise Python's support for > multiple inheritance. Java's all the less for not having it. However, > Java's facility for interfaces is a pure blessing, and I lack a > counterpart for it in Python. Please make up you mind. First you call encapsulation 'baby sitting', then you praise interfaces, which are precisely what encapsulation provides. > Java's interfaces were designed as a replacement for multiple > inheritance. The lack of support for multiple inheritance sometimes > makes design in Java grossly unintuitive, however. Object orientation is what makes Java counter-intuitive. Python, in many ways, is NOT restricted by a static type system, and therefore not restricted by a broken one like Java's. Instead, python programming is based on notions of protocols, such as 'sequence', where a mix of lightweight functional, procedural, and object oriented programming techniques coupled with dynamic typing, provide highly flexible notions of interfaces, not represented in the language (as they are in Java, Eiffel, and typically in C++ as well). One would hope for a decent static type system to be invented which can assist in typing such a protocol driven system, but it is not easy, and Python programmers prefer to live with notional interfaces and documentation. Note that a synthesis involving optional static typing is being worked on. In many senses, java is a lousy compromise. It doesn't have the high performance of C++, and it is just as hard to port serious systems as any other language. It is slightly simpler than C++, but not nearly as easy to use as python. java does have one advantage though: a serious GUI (Swing). If there is a reason for using Java, that is it. It's a mess. But it can do a lot more than Tk, and is more stable tham GTK. -- John (Max) Skaller, mailto:skaller@maxtal.com.au 10/1 Toxteth Rd Glebe NSW 2037 Australia voice: 61-2-9660-0850 checkout Vyper http://Vyper.sourceforge.net download Interscript http://Interscript.sourceforge.net From rhs@MIT.EDU Tue Apr 4 20:38:44 2000 From: rhs@MIT.EDU (Rafael H Schloming) Date: Tue, 04 Apr 2000 15:38:44 -0400 Subject: [Compiler-sig] Polymorphism/encapsulation in Python In-Reply-To: Your message of "Tue, 04 Apr 2000 23:20:04 +1000." <38E9EC04.B936EB90@maxtal.com.au> Message-ID: <200004041938.PAA06230@nerd-xing.mit.edu> > Object orientation is what makes > Java counter-intuitive. Python, in many ways, is NOT restricted by > a static type system, and therefore not restricted by a broken > one like Java's. Instead, python programming is based on notions of > protocols, such as 'sequence', where a mix of lightweight functional, > procedural, and object oriented programming techniques coupled with > dynamic typing, provide highly flexible notions of interfaces, > not represented in the language (as they are in Java, Eiffel, > and typically in C++ as well). What exactly is the difference between protocals and interfaces? I always looked at python as a language that makes very heavy use of interfaces and therefore just about any code you write to operate on lists or strings ends up working on anything satisfying the sequence interface. Personally I think making this explicit would be a great idea. > One would hope for a decent static type system to be invented > which can assist in typing such a protocol driven system, > but it is not easy, and Python programmers prefer to live with notional > interfaces and documentation. Note that a synthesis involving optional > static typing is being worked on. I've written large scale software in python that had very complicated class heirarchies with litteraly thousands of different classes. I think python scaled very well to this level and I would choose it over any other language, but what I found myself doing was putting in an assert for every single argument in order to catch little slips like swapping two arguments or passing in the wrong value somewhere. Without these asserts the system would have been difficult to debug because the errors wouldn't show up until very far beyond the place where the actual bug occured. An interface mechanism would have allowed me to express my asserts much more tersely and would have made my code far more generic since I would be asserting things like arg_foo satisfies interface bar instead of isinstance(arg_foo, ClassBar) where ClassBar is just a specific implementation of interface bar. --Rafael H. Schloming From DBinks@scigen.co.uk Wed Apr 5 09:06:22 2000 From: DBinks@scigen.co.uk (Binks, Dominic) Date: Wed, 5 Apr 2000 09:06:22 +0100 Subject: [Compiler-sig] Polymorphism/encapsulation in Python Message-ID: <554F5FFF12D4D311B04700508B5A9D291CF322@HARSTONMAIL> I would very much agree with this having used Python embedded. I always had to put exception handling round the main function/method called in order to catch problems. Often these could have been caught by some static type check tool. Even if the static types/interfaces just generated asserts for you that would be a real help. Dominic Binks Scientific Generics Ltd. -----Original Message----- From: Rafael H Schloming [mailto:rhs@MIT.EDU] Sent: 04 April 2000 20:39 To: John Max Skaller Cc: Ludvig Svenonius; compiler-sig@python.org; guido@python.org Subject: Re: [Compiler-sig] Polymorphism/encapsulation in Python I've written large scale software in python that had very complicated class heirarchies with litteraly thousands of different classes. I think python scaled very well to this level and I would choose it over any other language, but what I found myself doing was putting in an assert for every single argument in order to catch little slips like swapping two arguments or passing in the wrong value somewhere. Without these asserts the system would have been difficult to debug because the errors wouldn't show up until very far beyond the place where the actual bug occured. An interface mechanism would have allowed me to express my asserts much more tersely and would have made my code far more generic since I would be asserting things like arg_foo satisfies interface bar instead of isinstance(arg_foo, ClassBar) where ClassBar is just a specific implementation of interface bar. --Rafael H. Schloming _______________________________________________ Compiler-sig mailing list Compiler-sig@python.org http://www.python.org/mailman/listinfo/compiler-sig From timdowns@hotmail.com Wed Apr 5 09:52:19 2000 From: timdowns@hotmail.com (Timothy Downs) Date: Wed, 05 Apr 2000 01:52:19 PDT Subject: [Compiler-sig] Python Bytecode Message-ID: <20000405085219.31843.qmail@hotmail.com> I'm interested in writing a Python Virtual Machine, which processes the Python Bytecode (ie, the *pyc files). However, I cannot work out the format from the python source :(. Just wondering if anyone has any information of the format for the pyc files, for example, when the empty .py file produces a 100 byte or so pyc file. Information of bytecode programming, Python, or otherwise would be very hand. (BTW. yes i have read the Python standard docs (dis.dis)) Thanks Heaps timdowns@hotmail.com Tim Downs ______________________________________________________ Get Your Private, Free Email at http://www.hotmail.com From mwh21@cam.ac.uk Wed Apr 5 10:12:44 2000 From: mwh21@cam.ac.uk (Michael Hudson) Date: 05 Apr 2000 10:12:44 +0100 Subject: [Compiler-sig] Python Bytecode In-Reply-To: "Timothy Downs"'s message of "Wed, 05 Apr 2000 01:52:19 PDT" References: <20000405085219.31843.qmail@hotmail.com> Message-ID: "Timothy Downs" writes: > I'm interested in writing a Python Virtual Machine, which processes the > Python Bytecode (ie, the *pyc files). > > However, I cannot work out the format from the python source :(. > Just wondering if anyone has any information of the format for the pyc > files, for example, when the empty .py file produces a 100 byte or so pyc > file. > > Information of bytecode programming, Python, or otherwise would be very > hand. > > (BTW. yes i have read the Python standard docs (dis.dis)) > The *.pyc files are 8 bytes of header (a magic number and a time stamp, though I can't remember which order...) followed by a marshalled code object. So if you wan't to have a completely independent VM, you first need to write a demarshaller ... documentation is probably the source of Modules/marshal.c. Cheers, M. -- well, take it from an old hand: the only reason it would be easier to program in C is that you can't easily express complex problems in C, so you don't. -- Erik Naggum, comp.lang.lisp From guido@python.org Wed Apr 5 15:09:15 2000 From: guido@python.org (Guido van Rossum) Date: Wed, 05 Apr 2000 10:09:15 -0400 Subject: [Compiler-sig] Python Bytecode In-Reply-To: Your message of "05 Apr 2000 10:12:44 BST." References: <20000405085219.31843.qmail@hotmail.com> Message-ID: <200004051409.KAA16082@eric.cnri.reston.va.us> > The *.pyc files are 8 bytes of header (a magic number and a time > stamp, though I can't remember which order...) followed by a > marshalled code object. So if you wan't to have a completely > independent VM, you first need to write a demarshaller ... > documentation is probably the source of Modules/marshal.c. If you get JPython, there's marshalling code written in Python that may be easier to read than the C code. Also, get Jeremy Hylton's compiler (see this sig's homepage or archives), he has routines for generating Python bytecode files written in Python. --Guido van Rossum (home page: http://www.python.org/~guido/) From jeremy@cnri.reston.va.us Wed Apr 5 17:34:02 2000 From: jeremy@cnri.reston.va.us (Jeremy Hylton) Date: Wed, 5 Apr 2000 12:34:02 -0400 (EDT) Subject: [Compiler-sig] Python Bytecode In-Reply-To: <200004051409.KAA16082@eric.cnri.reston.va.us> References: <20000405085219.31843.qmail@hotmail.com> <200004051409.KAA16082@eric.cnri.reston.va.us> Message-ID: <14571.27386.329260.379670@goon.cnri.reston.va.us> >>>>> "GvR" == Guido van Rossum writes: GvR> Also, get Jeremy Hylton's compiler (see this sig's homepage or GvR> archives), he has routines for generating Python bytecode files GvR> written in Python. The code I have generates code objects using the new module, then puts the code objects into a pyc file using the marshal module. This approach only works if you already have a Python interpreter with access to these modules. It sounds like the JPython code is a better place to start. Jeremy From skaller@maxtal.com.au Wed Apr 5 17:48:37 2000 From: skaller@maxtal.com.au (John Max Skaller) Date: Thu, 06 Apr 2000 02:48:37 +1000 Subject: [Compiler-sig] Polymorphism/encapsulation in Python References: <200004041938.PAA06230@nerd-xing.mit.edu> Message-ID: <38EB6E65.D305C9EC@maxtal.com.au> Rafael H Schloming wrote: > > > Object orientation is what makes > > Java counter-intuitive. Python, in many ways, is NOT restricted by > > a static type system, and therefore not restricted by a broken > > one like Java's. Instead, python programming is based on notions of > > protocols, such as 'sequence', where a mix of lightweight functional, > > procedural, and object oriented programming techniques coupled with > > dynamic typing, provide highly flexible notions of interfaces, > > not represented in the language (as they are in Java, Eiffel, > > and typically in C++ as well). > > What exactly is the difference between protocals and interfaces? A protocol is a set of rules for using components together. Usually these rules are written down in news articles, documentation, etc: these things are partly designed, and partly established by practice with experience of using certain techniques. An interface is a way of _enforcing_ part of a protocol. For example, there is no encapsulation in Python: there are no interfaces to enforce it. However, you can program with the convention that private methods start with an underscore (_), and that you will NEVER set or get data attributes from a class instance. You TELL people using your class not to do it 'or on their head be it' :-) > I always looked at python as a language that makes very heavy use of > interfaces This is not really correct in Python. [Although the syntax is vaguely related to the protocols .. this is true in any language :-] Strangely, it IS correct when writing Python _extensions_. In this case, some of the protocols are embodied in generic functions of the C API: have a look at the documentation on the C API and notice it is divided into levels such as 'abstract objects layer', etc. > and therefore just about any code you write to operate on > lists or strings ends up working on anything satisfying the sequence > interface. Personally I think making this explicit would be a great > idea. What you say is common, and when you say 'making it explicit' I think you mean 'with language constructions (such as interfaces)'. The implication of these constructions is _enforcement_. Aka 'baby sitting' :-) Secondly, I agree with you that making it explicit would be a good idea. If only we knew how to do this. The problem is, we don't. To learn about interfaces, you must use a 'well-typed' programming language like ML. I recommend ocaml or haskell. You cannot learn about this from silly stuff like java. Actually, Python is also good, because you can experiment with 'typing ideas', without enforcement getting in the way. Consider the following python function: def f(x): f Exactly how would you 'type' this function f? This function is not useful. But the following one, in C++, truly is: ptm continuation() { ... } where the type of the function 'continuation' is 'pointer to member of the type of continuation'. You may wonder on the use of this function. It is the function which drives an callback driven interpreter. (you call some method on an object, and it returns the next method to execute: this method has the same type as the method returning it). > I've written large scale software in python that had very complicated > class heirarchies with litteraly thousands of different classes. I > think python scaled very well to this level From your description I'd guess this would be about the end of it's scalability. > and I would choose it over any other language, I wouldn't, unless Python offered some special advantage. My own 'medium' sized python program interscript -- which is much smaller than yours -- has reached the point where the lack of static typing and more powerful constructions, has made the code very hard to maintain. > but what I found myself doing was putting in an > assert for every single argument in order to catch little slips like > swapping two arguments or passing in the wrong value somewhere. > Without these asserts the system would have been difficult to debug > because the errors wouldn't show up until very far beyond the place > where the actual bug occured. An interface mechanism would have > allowed me to express my asserts much more tersely and would have made > my code far more generic since I would be asserting things like > arg_foo satisfies interface bar instead of isinstance(arg_foo, ClassBar) > where ClassBar is just a specific implementation of interface bar. No one disagrees with this. The question is how to do it. I have my own 'python' interpreter, (Vyper) which is considerbly more powerful than Python. It has superior static control, for example: fully nested scoping, 'compiling' all static variables [CPython does this now, but only for function locals] It has superior techniques for interface specification: nested modules, hiding of temporaries using with/do blocks, renaming imported symbols (import x as y), pattern matching, Greg Stein's typecheck operator x ! t, both dynamically, and also in function declarations where there is opportunity for compile time type checking (although I do not yet do this in Vyper). These abilities are not the same as a full scale static type system. But they permit _more_ checking at compile time, while hopefully not preventing uncluttered expression, nor generating weird error messages (symptoms of systems which require everything be declared (C++), and those using type inference to deduce types, respectively). -- John (Max) Skaller, mailto:skaller@maxtal.com.au 10/1 Toxteth Rd Glebe NSW 2037 Australia voice: 61-2-9660-0850 checkout Vyper http://Vyper.sourceforge.net download Interscript http://Interscript.sourceforge.net