From jim@Digicool.com Sat Nov 21 18:17:20 1998 From: jim@Digicool.com (Jim Fulton) Date: Sat, 21 Nov 1998 18:17:20 +0000 Subject: [Types-sig] Re: PRE-PROPOSAL: Python Interfaces References: <3642E80C.4A997D94@digicool.com> <3oE02.5$Hg.47@news-dal.corridex.com> <72gcjo$je1$1@thetenth.astat.de> <364c579e.516129164@news.triode.net.au> <72soaa$j2e$1@Starbase.NeoSoft.COM> <36528f91.923668414@news.triode.net.au> Message-ID: <365703B0.F1FBB2FD@digicool.com> Note that this discussion should move to the Types SIG: http://www.python.org/sigs/types-sig/. I am cross-posting to types-sig@python.org. Replies should be made to types-sig and omit comp.lang.python. John (Max) Skaller wrote: > (snip) > Next: examine the protocols module in interscript.core. > [It itself has a documented protocol -- a meta protocol?] > > It attempts to associate a tag with a documented protocol > specification, and lets you attach the tag to objects obeying that > protocol; and thus do run time tests for protocol adherence. > > This is _not_ the full scale category theory based > solution for Python: but it is a practical start to experiment > with protocols that doesn't require any changes to python, > and can support all the existing objects and protocols .. > including all third party ones -- without any changes to them. > [to see how .. look at the module!] > > Here is an example that conforms to the protocols > protocol :-) > > # a 'sink' is an object that support a writeline > # method that ...... > class Sink: > __class_protocols__ = ['sink'] > def writeline(self): > ... > > from interscript.core.protocols import has_protocol > def writefile(data, output): > if has_protocol(output, 'sink'): > output.writeline(data) > elif has_protocol(output, 'file'): > output.write(data + '\n') > elif has_protocol(output, 'string'): > file = open(output, 'a') > file.write(data + '\n') > file.close() > else: > raise "output must be filename, open file, or sink" > > The 'writefile' function is generic (polymorphic) in the sense that > it can be called with a sink, an open file, or a filename. > NOTE: a 'file' is not just a python file object. It is any object > with a write method -- which claims to be a file. > Similarly, a 'sink' is not an instance of class Sink: > it's any object with a suitable writeline() method, that claims > to be a sink. > > There is a nasty 'case switch' in the module. THIS IS BAD!!! > But what is good is that it switches on protocol -- not on type. > So it is 'an order of magnitude' better than type casing, > even if the technology is still wrong. > > What this example shows is that 'writefile' is in some sense has a protocol, > related to the 'sink', 'file', and 'string' protocols. > > To get rid of the case switch, what we do we need? > > I think we need protocol categories to define the relationships > between protocols. I'm working on it. :-) It looks to me like your protocols idea is pretty close to the "scarecrow" proposal, which is sketched out in: http://www.foretec.com/python/workshops/1998-11/dd-fulton-sum.html. I think that, at lease at the start, we are on the same wavelength. Some differences between the "scarecrow" proposal and your protocols idea: - in the scarecrow proposal classes need not inherit interface (protocol) conformance from base classes, and - interfaces are objects, not just string tags. Note that interfaces are arranged in a separate hierarchy (from classes), which perhaps allows them to provide the categorization you talk about above. Jim -- Jim Fulton mailto:jim@digicool.com Technical Director (540) 371-6909 Python Powered! Digital Creations http://www.digicool.com http://www.python.org Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats. _______________________________________________ Types-SIG mailing list Types-SIG@python.org http://www.python.org/mailman/listinfo/types-sig From gmcm@hypernet.com Sat Nov 21 23:24:11 1998 From: gmcm@hypernet.com (Gordon McMillan) Date: Sat, 21 Nov 1998 18:24:11 -0500 Subject: [Types-sig] Re: PRE-PROPOSAL: Python Interfaces In-Reply-To: <365700BD.FE8B1776@technologist.com> Message-ID: <1300449328-21246608@hypernet.com> [I wrote] > > I wholeheartedly agree with Paul here. But I notice that his real > > concern (like mine) appears to be "performance". I know that Roger > > (and, I suspect, Jim F) are more concerned with "safety". [Paul 'the Politico' Prescod prevaricates] > Actually, I am equally interested in the two halves. I just know how > to tailor my message for the audience. :) > > The rest of this message isn't really oriented towards you, Gordon. > I'm just venting. That's alright. I'll rip your arms off anyway . > I don't entirely understand how there can be people who really love > Python and yet are against the idea of interfaces. What is the > "sequence protocol" if not a language-builtin interface? What is the > "mapping protocol"? etc. I don't understand why someone would object > to generalizing this fantastically successful idea. I don't object, but I worry. For example, the "sequence protocol" is a very fuzzy thing. If you start with the abstract sequence protocol (as defined by the C API), only List is an implementation of which B. Meyer would approve, (SetItem is in the abstract interface, for example). OK, so lets pretend that's cleaned up. Now we're getting methods on List so it can be treated like a Stack. Stack is certainly not another name for List, nor for Sequence (whatever that means). It's another protocol, right? But it's one that is automatically implemented if you implement the List protocol. How about FileInput? That's a sequenceish protocol. A read-only, forward-only, sequential-only sequence. How do we (sensibly and comprehensibly) capture the fact that most sequenceish protocols will suffice where a FileInput protocol is required, with the exception of those stack-ish style sequences? In fact (as Barry was supposed to point out at the conference for me, harrumph, harrumph), the set of all interfaces / protocols is the set of all subsets of the set of all signatures - without taking behavior into account. That is, the set of all interfaces has a cardinality that I don't particularly want to contemplate. At the moment "sequence protocol" means 2 different things. It means Py_SequenceCheck(o) == TRUE (which I hope the type / class unification fixes), and it means that if you squint a little and wave your hands alot, this thing could be considered a sequence. My worry is that if you attempt to make the latter definition more precise, you in fact increase complexity. I know, worrying is Tim's job, not mine. but-he-was-in-Vegas-discovering-the-Meaning-of-Life-ly y'rs - Gordon From mal@lemburg.com Sun Nov 22 10:51:13 1998 From: mal@lemburg.com (M.-A. Lemburg) Date: Sun, 22 Nov 1998 11:51:13 +0100 Subject: [Types-sig] Scarecrow Proposal Message-ID: <3657ECA1.6B2BD52@lemburg.com> I've just read the Scarecrow Proposal and have a few questions (sorry, if these have already been answered at the conference... I haven't been there): · Why is it important that classes and interfaces are different ? Is this just to make sure you can't mix classes and interfaces in __bases__ ? IMHO, we should use as much as possible from the existing features, so I think building the interface hierarchy with plain Python classes and then sticking them into the class hierarchy as __implements__ (should probably be named __interfaces__; a builtin function implements() could take care of the actual lookup in whatever is suitable). If there are strong reasons for making the two different in the sense that type(class) is not type(interface), then I'd opt for simply cloning the class implementation by creating a second type object that is simply a copy of the class' one. · Would this proposal get us closer to static typing ? I'm thinking of a way to declare a named object ABC as having an interface XYZ (e.g. a function could be assigned an interface stating that the first argument must be an integer). Since the interface hierarchy is something that is built dynamically, the compiler would have to be given two things: an already executable script INT that builds the interface hierarchy and the source script SRC referring to this hierarchy. It could then optimize e.g. the function calls to abc knowing that the first argument is an integer and also add checks at C level to ensure this (a PyInt_Check() is a whole lot faster than the same check via type() in Python). A general syntax would have to be introduced to accomplish this (which would have to go into Python 2.0 I presume), e.g. def ABC(a,b,c) implements TypedArguments(Integer,Any,Any): a = a + 1 return a, b+c where TypedArguments is a helper function defined in the INT script which creates an interface for the given argument combination (Integer and Any would also have to be defined in INT and the compiler could call methods on these to find out how to optimize them). Note: The helper function would have to be called at *compile time*. Also note: The code INT could have been imported by the compiler as module prior to compilation, e.g. if there were a standard hierarchy available no extra fuzz would be implied for the compilation step. · In any case, is the interface proposal going to effect any existing code ? I would like to see it implemented as additional useful feature, but in a way that is orthogonal to the existing implementation. Especially: no performance hit if you don't use it. · Should interfaces also allow changing the method invokation behaviour ? E.g. pre- and post-conditions could be defined in the interface rather than in a meta-class. When the compiler sees a method ABC_precondition in the interface it generates code to call this function prior ABC. Same for ABC_postcondition. Do these Eiffel constructs belong into interfaces ? IMHO, they do. -- Marc-Andre Lemburg Y2000: 404 days left --------------------------------------------------------------------- : Python Pages >>> http://starship.skyport.net/~lemburg/ : --------------------------------------------------------- From papresco@technologist.com Sun Nov 22 20:03:07 1998 From: papresco@technologist.com (Paul Prescod) Date: Sun, 22 Nov 1998 14:03:07 -0600 Subject: [Types-sig] Re: PRE-PROPOSAL: Python Interfaces References: <1300449328-21246608@hypernet.com> Message-ID: <36586DFB.C79CF6E6@technologist.com> Gordon McMillan wrote: > > I don't object, but I worry. For example, the "sequence protocol" is > a very fuzzy thing. Let me see if I understand your concern: imagine that you have code that does not require the full List interface. You only require some sub-part of it - read-only list. A client has code that implements read-only list. Under the current system, the client programmer could read the docs or code, take note that the read-only list is good enough, and pass it in. The interface is not checked by the computer, so we can work around the holes in the interface hierarchy by adopting out-of-band conventions. In the C++ templates world, the compiler would do static interface checking of the instantiation of the template and if nothing breaks, it will just work. But in the interface world, you can't express the compatibility between the client and the server in terms of the standard interfaces because read-only list is not standard, and it doesn't make sense for the Python standard library to try and express every variant of list. You could invent a brand new interface, but then *other clients* who just want to pass in a List would have a problem. So we would make their lives harder: they would have to wrap their lists in read-only-list wrappers, just to get around the type system! Let me suggest a way out of this quandry. As in John Skaller's protocols class, we must be able to assert that some existing interface or type conforms to a new interface. So your module would say: interface read_only_list: ...some subset of list.... list.add_interface( read_only_list ) Now we could also discuss the case where the required interface is a superset of any particular existing interfaces, but that's a problem under the current system also. If you need a list with a "push" method, then you've got to define a wrapper for the list that does "push". For instance, that's what the stringio class does. > How about FileInput? That's a sequenceish protocol. A read-only, > forward-only, sequential-only sequence. How do we (sensibly and > comprehensibly) capture the fact that most sequenceish protocols will > suffice where a FileInput protocol is required, with the exception of > those stack-ish style sequences? We may or may not be able to do that with type signatures or any other formalism, but we can certainly do it with prose. That's what C++ does: http://www.cs.rpi.edu/projects/STL/htdocs/node53.html As Jim Fulton has said, even this: interface FileInput: "This is a read-only, forward-only, sequential-only sequence." is better than lore shared on a newsgroup, which is what we have today. And even given that almost-vacuous declaration, this: def ReadLine( input ): InterfaceCheck( input, FileInput ) is better than just trusting that caller understood that the argument was supposed to be a FileInput type. And if we check method signatures, then we have gone even farther to prove that the caller understands what is going on. And if we can solve the halting problem, we could demonstrate beyond a shadow of a doubt that the caller knew exactly what he or she was doing. But as Skip says, let's put the halting problem aside for a moment. > In fact (as Barry was supposed to point out at the conference for me, > harrumph, harrumph), the set of all interfaces / protocols is the set > of all subsets of the set of all signatures - without taking behavior > into account. That is, the set of all interfaces has a cardinality > that I don't particularly want to contemplate. Right, and we need a way of talking about the subsets that we are actually interested in. And we need to be able to define those susets more or less "on the fly" when we need them, instead of building top-down. Paul Prescod - http://itrc.uwaterloo.ca/~papresco A traceback is better than an unadorned exception. An exception is better than a naked core dump. A core dump is better than silent failure. Success beats all. Is Python the next step up your language ladder? From tim_one@email.msn.com Sun Nov 22 22:00:25 1998 From: tim_one@email.msn.com (Tim Peters) Date: Sun, 22 Nov 1998 17:00:25 -0500 Subject: [Types-sig] RE: PRE-PROPOSAL: Python Interfaces In-Reply-To: <1300449328-21246608@hypernet.com> Message-ID: <000201be1663$827f3260$2f9e2299@tim> [Paul Prescod] > I don't entirely understand how there can be people who really love > Python and yet are against the idea of interfaces. What is the > "sequence protocol" if not a language-builtin interface? What is the > "mapping protocol"? etc. I don't understand why someone would object > to generalizing this fantastically successful idea. [Gordon McMillan] > I don't object, but I worry. For example, the "sequence protocol" is > a very fuzzy thing. Indeed it is! And e.g. I have any number of "sequence types" that implement only __getitem__, and even then only for purposes of the for/in protocol: ignoring __getitem__'s argument, just pumping out "the next one" each time __getitem__ is called. Abstractly these types are really just ForInImplementers, but because we have no formal rules (neither names known to Python nor even accepted conventions) now, your generic function is likely to try hasattr(timsThing, "__getitem__") and draw wrong conclusions from the result. > If you start with the abstract sequence protocol (as defined by the C > API), only List is an implementation of which B. Meyer would approve, > (SetItem is in the abstract interface, for example). > > OK, so lets pretend that's cleaned up. Now we're getting methods on > List so it can be treated like a Stack. Stack is certainly not > another name for List, nor for Sequence (whatever that means). It's > another protocol, right? But it's one that is automatically > implemented if you implement the List protocol. Under the proposal, though, conformance isn't a matter of checking structural equivalence of method collections, it's a matter of looking for specific interface objects in constrained places. So if you have class Stack(Interface): def push(self, thing): MysteryGoesHere def pop(self): MysteryGoesHere then it's simply consequence-free coincidence if "class List(Interface)" contains methods with matching names and signatures -- unless List.__implements__ contains specifically Stack. IOW, Lists do *not* implement the Stack protocol just because "it looks like" they do -- if they do at all, it's only by virtue of List explicitly saying it implements specifically Stack. Today, though, there's no way to draw the distinction at all. OTOH, unlike e.g. JimF I have nothing against stuffing some default implementation into interfaces. To the contrary, if I *wanted* to say that all List implementers implement Stack too, I think class List(Interface, Stack): #??? spelling of interface __implements__ = Stack #??? ... hierarchy is unclear def pop(self): raiseDeferredError() def append(self, thing): raiseDeferredError() ... def push(self, thing): self.append(thing) is the obvious way to do it. This doesn't preclude a different implementation of push, it simply reflects the truth that List *can* give you an implementation of push guaranteed to meet push's spec, assuming only that the implementations of List.append and List.pop meet *their* specs. For a simpler example, after rich comparisons are in class SaneOrdering(Interface): def __eq__(self, other): raiseDeferredError() def __ne__(self, other): return not self.__eq__(other) would save 99% of the SaneOrderings ever created the pointless and error-prone bother of implementing __ne__ themselves. That is, if the post-condition for SaneOrdering.__ne__(self, other) is assert Result == not self.__eq__(other) what better way to encourage correct software than to provide a default implementation that guarantees it without further ado? Many methods in rich interfaces exist simply for user convenience, having obvious (but clumsy to type out) implementations in terms of other methods. > How about FileInput? That's a sequenceish protocol. A read-only, > forward-only, sequential-only sequence. How do we (sensibly and > comprehensibly) capture the fact that most sequenceish protocols will > suffice where a FileInput protocol is required, with the exception of > those stack-ish style sequences? > > In fact (as Barry was supposed to point out at the conference for me, > harrumph, harrumph), the set of all interfaces / protocols is the set > of all subsets of the set of all signatures - without taking behavior > into account. That is, the set of all interfaces has a cardinality > that I don't particularly want to contemplate. I don't think the proposal is rich enough to capture everything people do in Python -- and don't think it needs to be. I'll attach my original essaylet on this topic, one of the 95 Fece^H^H^H^HTheses I nailed to the egroups door. The point of all this to me is to make the simple things simpler -- wrestling with anonymous folklore protocols is a real barrier in Python practice, and a *needless* one much of the time. > At the moment "sequence protocol" means 2 different things. It means > Py_SequenceCheck(o) == TRUE (which I hope the type / class > unification fixes), and it means that if you squint a little and wave > your hands alot, this thing could be considered a sequence. My worry > is that if you attempt to make the latter definition more precise, > you in fact increase complexity. I think you *do* increase complexity for experts (see below), but reduce it for everyone else. "Sequence" can be defined in an "overly restrictive" way and still leave 99% of people happy 99% of the time. > I know, worrying is Tim's job, not mine. I can't make enough time for it, though, so appreciate your help . On c.l.py you raised the specter of suddenly needing to mark all your classes as implementing other peoples' interfaces in order to use their modules at all, and that's something I worry about more now than I did when I wrote the attached. Python derives much power from "folklore protocols" now, and indeed that cannot be captured in full short of (as you say) giving at least one name to each member of the powerset of all signatures. good-thing-python-doesn't-restrict-identifier-length-ly y'rs - tim [point 14 from Timmy's egroups manifesto] 14. [folklore is purged from the core] Enough sensible builtin "marker classes" are provided so that the documentation for every builtin function and builtin method can, for every argument, say that (at least) instances of such-and-such a builtin class are accepted. Long note: I expect this to be disliked by someone who matters <0.7 wink>, but I think it's important. The "don't check the type, check the methods" philosophy of Python is very powerful, yet rarely needed in most peoples' code -- but gets in the way when it's not needed because there's no other choice but to use it. People simply don't know what "pass something that acts like a sequence" or "acts like a file" etc *means*, and it turns out an accurate answer is sometimes darned hard to give. In my own code I'd almost always be happy e.g. checking isinstance(x, Sequence) at the start of a function -- except lists, tuples and strings won't cooperate. If they have classes down the road, and I can change their classes' bases, I can insert my own "Sequence" marker in the builtins. But I believe enough other people would do likewise that we'd end up with a Tower of Babel. Python can cut that off, and simultaneously build a foundation for formal interfaces and optional "type" declarations, by taking a stand and treating classes in the core as if they meant something. This does not break current, or preclude people from writing new, hyper-general methods and functions using ad hoc mixtures of isinstance, hasattr, and "just try it and see whether it blows up". In fact, the C implementation still needs to be written that way. It does send a message that such convolutions support advanced capabilities, though, and usually aren't needed in everyday code. That in turn makes Python friendlier for all save a tiny elite. Short note: Peeking ahead in other position papers, I see that "execution context" (something suitable to pass to eval/exec) is yet another piece of folklore in the making. Boo! Good idea, but let's use classes to give it a real name (== one Python understands too). In fact, brand new features like that can *require* derivation from a specific builtin marker class, and so enjoy simpler & clearer documentation (albeit not necessarily simpler implementation). From tim_one@email.msn.com Sun Nov 22 22:00:21 1998 From: tim_one@email.msn.com (Tim Peters) Date: Sun, 22 Nov 1998 17:00:21 -0500 Subject: [Types-sig] RE: PRE-PROPOSAL: Python Interfaces In-Reply-To: <199811211454.JAA18077@eric.cnri.reston.va.us> Message-ID: <000001be1663$7fd08d20$2f9e2299@tim> [followups to types-sig@python.org, please] [Guido "Trained as a Mathematician!" van Rossum] > I just read Uche's objection against the current static type ideas. > He mentions that he much favors John Skaller's proposal. > > Unfortunately, much as I try to read Skaller's prose, I don't > understand a bit of it. Given your middle name (see above), perhaps that says something about the potential popular appeal of the approach -- I'm not sure "Teach Yourself Functors from Abstract Categories in 21 Days!" or "Secrets of the Morphism Masters!" are destined to top the SAMS best-seller list . I last gave close attention to approaches "like this" when they were called "algebraic specification of abstract datatypes" in the late 70s and early 80s. Back then it was neat stuff and generated some interesting results. But having cast programming into a formal system the temptation was to forget the origin and play with the formal system for its own sake; later efforts to reflect insights from the latter back into language design were usually greeted with about as much enthusiasm as you'd expect given John's "What is an instance? Any other category X which is the image of a functor from [the abstract category corresponding to the thing you (in your charming naivete ) call "the class" today]". Now *maybe* dumping category theory into the mix has profoundly altered this disconnect from everyday practice -- but to judge from John's presentation, it appears to have gotten worse. > The only impression I get is that perhaps there's a counter-proposal > in there to consider; however I have no idea what it is. > > Did anybody have more success in understanding it? If so, I'd love it > if you could post a summary... I believe PaulP and JimF have both invited John to comment on the Scarecrow Proposal, and it seems to me too that the SP *appears* to have a lot in common with the protocols module John created for Interscript (the only thing of his on the table I think qualifies as a concrete counter-proposal). in-theory-theory-is-great-ly y'rs - tim From guido@CNRI.Reston.VA.US Mon Nov 23 01:00:34 1998 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Sun, 22 Nov 1998 20:00:34 -0500 Subject: [Types-sig] Re: PRE-PROPOSAL: Python Interfaces In-Reply-To: Your message of "Sun, 22 Nov 1998 17:00:21 EST." <000001be1663$7fd08d20$2f9e2299@tim> References: <000001be1663$7fd08d20$2f9e2299@tim> Message-ID: <199811230100.UAA20113@eric.cnri.reston.va.us> I think I understand John Skaller's argument better now. He's objecting against the threat that (e.g.) functions which work fine with any kind of -- abstractly defined -- integers might be littered with 'int' declarations all over, restricting them to 32-bit machine ints. His counterproposal (protocols) is very similar to Jim Fulton's interfaces proposal, as many have noticed. There is probably some truth in his fear -- those who like to add type annotations for performance reasons (e.g. Jim Hugunin) would like to annotate code with type declarations that will allow a compiler to generate the "obvious" Java code. I think we can dream up a sufficiently rich set of interfaces to make everyone happy. For example, here's a bunch of numeric interfaces. I envision that some of these can be combined to form various concrete types. (We could also look at Scheme's numerical tower for inspiration.) 0. number: it is assumed that numeric types are embedded in each other in some way (this is the Scheme numerical tower I think); they support operators + - *, usually / and %, perhaps **, and possibly & | ~ ^ << >> (it might make sense to define different interfaces depending on which operators are supperted; some of the categories below probably assume a certain set of operations) 1. complex vs. real: real -- any subset of the real numbers (this includes ints and rationals) complex -- has re and im fields (which are themselves real) (Mike McLay will love the fact that this can be extended to include quaternions and even higher-order numeric types) 2. exact vs. inexact: exact -- e.g. integers and rationals; also fixed point inexact -- floating point arithmetic 3. subdivisions of exact: rational (arbitrary precision) long (arbitrary precision) int (32 bits) short (16 bits) byte (8 bits) bit (1 bit) fixed_decimal -- a signed fixed decimal number with n digits before and m digits after the decimal point 4. signed vs. unsigned: for the fixed-size binary size integer types it may make sense to recognize unsigned variants; these are effectively rings module 2**N 5. overflow behavior (only for fixed-size ints): checked overflow -- raises OverflowError when results don't fit unchecked overflow -- take results module 2**N (for example) 6. floating point (these are just examples) float -- 32bit float double -- 64bit float extended -- 80bit float Other categorizations are possible too (not clear if I'm stretching the mechanism too far here): - division behavior: - exact result (rationals) - approximate result (inexact, fixed_decimal) - truncate towards zero (C/Java style ints) - truncate towards -inf (Python style 32-bit ints) - undefined (members of true rings) - bit shift and mask operations: - signed shifts - unsigned shifts - circular shifts - undefined Shifting gears, we can come up with a similar family of container and string types (my colleagues at CNRI and Greg Stein will recognize some of this from my scribbles on the whiteboard last Friday). The quoted section from the C++ standard template library also applies here. container: - anything supporting __getitem__; probably also __len__ mutable vs. immutable (subclass of container): - mutable has __setitem__ and perhaps __delitem__ - immutable may have __hash__ and comparisons sequence vs. mapping (subclass of container): - sequences require keys to be ints and support slicing - mappings don't Note that there could be a read-only mapping type. I propose to add specific interfaces (perhaps "pseudolist" and "pseudodict"?) for types that support all the additional methods that lists resp. dictionaries have (append, sort, reverse, index, __in__ and some more for lists; has_key, keys, items, values, update, copy for dictionaries). I don't even mind if there were (obscurely-named) names for interfaces specifying some subsets of these, e.g. for a dict that supports has_key and keys but not items or values, etc. Gotta run, --Guido van Rossum (home page: http://www.python.org/~guido/) From gmcm@hypernet.com Mon Nov 23 03:17:41 1998 From: gmcm@hypernet.com (Gordon McMillan) Date: Sun, 22 Nov 1998 22:17:41 -0500 Subject: [Types-sig] Re: PRE-PROPOSAL: Python Interfaces In-Reply-To: <36586DFB.C79CF6E6@technologist.com> Message-ID: <1300348907-542558@hypernet.com> [Gordon] > > I don't object, but I worry. For example, the "sequence protocol" is > > a very fuzzy thing. [Paul] > Let me see if I understand your concern: imagine that you have code > that does not require the full List interface. You only require some > sub-part of it - read-only list. A client has code that implements > read-only list. Under the current system, the client programmer > could read the docs or code, take note that the read-only list is > good enough, and pass it in. The interface is not checked by the > computer, so we can work around the holes in the interface hierarchy > by adopting out-of-band conventions. That last sentence is a sneaky piece of rhetoric. More accurate, and almost as misleading, to say that the required interface has been completely spelled out by the methods that are called. > In the C++ templates world, the compiler would do static interface > checking of the instantiation of the template and if nothing breaks, > it will just work. This is actually a criticism of C++ templates. Nowhere is it spelled out that "tempate blah blah" requires that T implement spam_spam_eggs_and_spam(). You find out by reading the source. Or if you're like me, by reading the error message. Which is the way I find out, in Python, that the object I passed in doesn't do the job. Java interfaces spell it out, but make unwarranted assumptions about inheritance, and don't generate code. Of course, a C++ template generates code to handle what we Pythoneers can do directly, so we don't need to generate code. (Yet.) > ... it doesn't > make sense for the Python standard library to try and express every > variant of list. That's a fair statement of concern #1. Concern #2 is that it's really only "optional" if I can use your function (which wants an object which implements the ReEnactsPearlHarborInAMudField interface) with any old thing I choose, and take my lumps at runtime. [snip snip - I bring up another odd-ball "interface"] > We may or may not be able to do that with type signatures or any > other formalism, but we can certainly do it with prose. That's what > C++ does: > > http://www.cs.rpi.edu/projects/STL/htdocs/node53.html I don't think STL makes a good model for anything we are trying to do. For one thing, while STL is very powerful, it actually makes some simple things harder. For another, most of it's power is in making a statically typed language behave (as far as coding is concerned) like a dynamically bound one. Mostly, STL generates code so you can write typeless templates and get fully typed instanciations. Maybe we'll need that in SPython, but not in Python. > As Jim Fulton has said, even this: > > interface FileInput: > "This is a read-only, forward-only, sequential-only sequence." > > is better than lore shared on a newsgroup, which is what we have > today. I realize fully that this is a strong argument for many. It's not for me. And a deft twist of Occam's Razor might yield the conclusion that an EmptyDocStringException is a better solution . > Right, and we need a way of talking about the subsets that we are > actually interested in. And we need to be able to define those > susets more or less "on the fly" when we need them, instead of > building top-down. I'll buy that conclusion. I'm about 50% playing devil's advocate, and trying to make sure people know what they're buying. But the other 50% of me is serious. When I write a GUI app, for example, I'm all in favor of optimization (and safety) on the data / "application" logic side. But I tend to use heavily dynamic GUI code. There's no reason to optimize GUI code. As for "safety": [out of order snippet]: > is better than just trusting that caller understood that the > argument was supposed to be a FileInput type. And if we check > method signatures, then we have gone even farther to prove that the > caller understands what is going on. And if we can solve the > halting problem, we could demonstrate beyond a shadow of a doubt > that the caller knew exactly what he or she was doing. But as Skip > says, let's put the halting problem aside for a moment. "caller understands", "caller knows", well excuuuuuse me, but I'm the caller, and if I can trick your code into doing what I want even if it's not what you expected, then more power to me! and-a-specific-solution-to-the-halting-problem-is-found-when -it-doesn't-blow-up-in-my-face-ly y'rs - Gordon PS And yes, I know you'll never find the general solution this way... From gmcm@hypernet.com Mon Nov 23 03:45:20 1998 From: gmcm@hypernet.com (Gordon McMillan) Date: Sun, 22 Nov 1998 22:45:20 -0500 Subject: [Types-sig] RE: PRE-PROPOSAL: Python Interfaces In-Reply-To: <000201be1663$827f3260$2f9e2299@tim> References: <1300449328-21246608@hypernet.com> Message-ID: <1300347251-642173@hypernet.com> [Gordon] > > OK, so lets pretend that's cleaned up. Now we're getting methods on > > List so it can be treated like a Stack. Stack is certainly not > > another name for List, nor for Sequence (whatever that means). It's > > another protocol, right? But it's one that is automatically > > implemented if you implement the List protocol. [Uncle Timmy] > Under the proposal, though, conformance isn't a matter of checking > structural equivalence of method collections, it's a matter of > looking for specific interface objects in constrained places. So if > you have > > class Stack(Interface): > def push(self, thing): MysteryGoesHere > def pop(self): MysteryGoesHere > > then it's simply consequence-free coincidence if "class > List(Interface)" contains methods with matching names and signatures > -- unless List.__implements__ contains specifically Stack. > > IOW, Lists do *not* implement the Stack protocol just because "it > looks like" they do -- if they do at all, it's only by virtue of > List explicitly saying it implements specifically Stack. Today, > though, there's no way to draw the distinction at all. Which brings up the point Paul made that if the all-omiscient Guido forgot to say that (the builtin) List implements Stack, and his time machine will have been in the body shop because of that fender-bender, then I'd better be able to say at runtime that "I know this is a List, but you'll treat it like a Stack anyway". > OTOH, unlike e.g. JimF I have nothing against stuffing some default > implementation into interfaces. Yeah. Somehow my C++ "abstract base classes" always end up not being abstract. [snip] > [point 14 from Timmy's egroups manifesto] Which Basilica door did you nail this to? (IE, where can I read it?) > 14. [folklore is purged from the core] Enough sensible builtin > "marker classes" are provided so that the documentation for every > builtin function and builtin method can, for every argument, say > that (at least) instances of such-and-such a builtin class are > accepted. [snip] > People simply don't know what "pass something that acts like a > sequence" or "acts like a file" etc *means*, and it turns out an > accurate answer is sometimes darned hard to give. I will admit to being completely mystified by the following (from lib/module-select.html): "You may also define a wrapper class yourself, as long as it has an appropriate fileno() method (that really returns a Unix file descriptor, not just a random integer)." particularly-on-WIndows-ly y'rs - Gordon From gmcm@hypernet.com Mon Nov 23 04:01:59 1998 From: gmcm@hypernet.com (Gordon McMillan) Date: Sun, 22 Nov 1998 23:01:59 -0500 Subject: [Types-sig] Re: PRE-PROPOSAL: Python Interfaces In-Reply-To: <199811230100.UAA20113@eric.cnri.reston.va.us> References: Your message of "Sun, 22 Nov 1998 17:00:21 EST." <000001be1663$7fd08d20$2f9e2299@tim> Message-ID: <1300346251-702304@hypernet.com> [Guido] > I think I understand John Skaller's argument better now. He's > objecting against the threat that (e.g.) functions which work fine > with any kind of -- abstractly defined -- integers might be littered > with 'int' declarations all over, restricting them to 32-bit machine > ints. His counterproposal (protocols) is very similar to Jim > Fulton's interfaces proposal, as many have noticed. There is > probably some truth in his fear -- those who like to add type > annotations for performance reasons (e.g. Jim Hugunin) would like to > annotate code with type declarations that will allow a compiler to > generate the "obvious" Java code. Jim (and someday, SPython) can already determine that a number used internally to a function may be optimized to a (primitive) int. Conceivably, if the arg is always used as a (primitive) int, it can be optimized to be converted only once. On much shakier ground (and with less payback) an abstract interface could still yield optimizations, at the cost of a core dump instead of an exception when violated. > I think we can dream up a sufficiently rich set of interfaces to > make everyone happy. I like the (elided) proposals, but some people are only happy when they're unhappy, (or they're making someone else unhappy ). Not pointing any fingers. well-maybe-one-ly 'yrs - Gordon From James.D-C@dial.pipex.com Mon Nov 23 09:26:14 1998 From: James.D-C@dial.pipex.com (James Dawes-Cooke) Date: 23 Nov 98 09:26:14 +0000 Subject: [Types-sig] Some comments. Message-ID: <8345.631T725T5661337@dial.pipex.com> Hi. While this SIG is still "setting the scene", I'd like to submit this msg which outlines my personal take on all this. Some of it is summarization of what others have said (or at least, my interpretation) and some of it forms the basis of a proposal (partly at the user-level, partly at the implementation). I apologise for it being in the form of a brain-dump, but I wanted to jump in now while the time was ripe. The "===" bars indicate a change of sub-topic while the "---" bars split the main thrust from the trailing comments (you'll see what I mean). ==== Classes and Interfaces are different First, I believe that the class hierachy should be distinct from the interface hierachy. Marc-Andre asked for reasons why this should be so. The simple reason is that the interface is an abstract identification of WHAT an object CAN DO whereas the class is a specific implementation of HOW an object does those things. The former is important to the caller of the methods in the objects belonging to the class, the latter is not. So, we can have objects from very different pedigrees (doing things differently) which can adhere to a common interface (are able to do the same thing in a consistent way). If you tie the two together, you can only build objects with interface I if you also inherit from the class C(I). This may not be appropriate. ==== Namespaces and "bound interfaces" One thing I haven't seen anyone mention yet is that of interface namespaces. Specifically, if I have two interfaces which both have a "frobnicate" method, how can I build a class which can act as either "type" ? If the interface methods could be specified as a second-level within a class, it would be possible to implement an object with two clashing interfaces: class C: interface I1: def frobnicate(self, arg1, arg2): pass interface I2: def frobnicate(self, arg): pass This would be possible without this construct with something like: class C implements I1, I2: def frobnicate(self, *args, **kw): if lookslikeI1(self, args, kw): apply(I1.frobnicate, ....) else: apply(I2.frobnicate, ....) However, the lookslikeI1() function may not be trivial to write, and forces run time checking forevermore. Furthermore, if the C class also implements a third interface, I3, which grows a "frobnicate" method at some point, the run time check above will do the wrong thing (either an I3 call looks sufficiently like an I1 call, or the default case is not properly handled). Besides, who actually implements lookslikeI1() ? Too much knowledge of "outside" would be needed if it were implemented within the I1 interface, and too much knowledge of interface internals would be required if it were implemented externally to the interface. If the former style of interface syntax were to be implemented, there would need to be a way of calling the correct frobnicate. I'll use the '!' operator as an "interface selection" operator for the sake of example. The "interface selection" operator binds the method call through a specific interface implemented by the class of the object: c = C() c!I1.frobnicate(1, 2, 3) c!I2.frobnicate("foo") This model also provides a type of coercion (or "bound interface"). The above example again ... c = C() ci1 = c!I1 ci2 = c!I2 ci1.frobnicate(1, 2, 3) ci2.frobnicate("foo") When you say "ci1 = c!I1" you are saying "let ci1 be an alias of c, but bound so that it uses the interface of I1 for all method calls". I like using "as" for the operator: c = C() ci1 = c as I1 ci2 = c as I2 This reads much better when the interfaces are called "Sequence" or "Stack". This "coercion" (or "selection" or "refinement") could also be applied to function parameters: def pop2(s as Stack): p = s.pop() return s.pop(), p What it's saying is "I don't care what object you pass me as long as I can treat it like a 'Stack'". I haven't thought much about how this relates to the static typing issues also being discussed, but I believe the two overlap. ---- The following isn't really important at this stage, but ... It looks a little ugly when more than one method is used. Assuming I1.frobnicate() returns an object which implements interface "I0": c!I1.frobnicate()!I0.method() The alternative: c as I1.frobnicate() as I0.method() is more pleasant, but doesn't look quite right because "as" has a higher precedence than "." whereas visually "I1.frobnicate" and "I0.method" look like basic units in this expression rather than "c as I1" and "..... as I0". ---- Footnote The above section is written from a user point of view with little thought into possible efficient implementation. My one thought on that front is that possibly the code: class C: interface I1: def frobnicate(self, arg1, arg2): pass ... should end up actually generating "__I1__frobnicate" style methods so things remain "flat" within the class. The "bound interface" would then be little more than a wrapper around __getattr__ to prefix the method lookup with the interface name. ==== "Pure" interfaces In John Skaller's message, he was wondering how to get rid of that "case" statement in his method, which decided which interface to use on an object. I suggest the possibility of using "pure interfaces". That is, an interface which can be implemented purely in terms of another interface which the system can then "apply" to any object which implements all the component interfaces. Something like: interface P requires I1, I2, I3: def foo(self, arg): return self!I1.frobnicate(), self!I2.spam(arg) def bar(self): return self!I3.parrot("dead") + I1.vroom(I2.volts(4000)) Using this "pure interface", I can def spangulate(obj as P): return obj.bar() / 2 It doesn't matter that "obj" wasn't defined as supporting interface P - interface P can deduce that it does in reality, because all the component parts are there. When "obj" is "bound" to the "P" interface with "obj as P", Python would ensure that the requirements list (I1, I2 and I3) are indeed implemented by "obj". Interface P is just a logical grouping of other interfaces and might exist just for my script or application. If it were possible to use logical connectives to specify the required interface list ("interface P requires I1 or (I2 and I3):"), then John's example could be simplified with a "pure interface": interface P requires I1 or I2 or I3: def foo(self, arg): if implements(self, I1): return I1.func(self, ....) if implements(self, I2): return I2.func(self, ....) return I3.func(self, ....) The bottom line is that this just moves the "case" from one place to another, but it means that a simple "as" operator makes the code available from many places. A function does that too, but you can not slot a function into an interface hierachy. ==== Syntactical anomoly Assuming the interfaces "Sequence" (as supported by lists/tuples) and "Mapper" (as supported by dictionaries) are available, there is a syntactical problem with the current item access operator "[]". Suppose I had a class, C, which implemented both interface "Sequence" and interface "Mapper". The purpose of my class is to store a dict with numeric keys. I need to be able to extract a specific entry from the dict (a Mapper operation) and I need to be able to iterate over the object in the order the dict entries were added (a Sequence operation). Where 'c' is an object of class C, the expression "c[0]" is ambiguous. Does it mean the item with ordinal 0 (a Sequence operation) or the item with key 0 (a Mapper operation) ? I have no way of knowing within my __getattr__ (and if we had "interface namespaces", Python has no way of knowing which __getattr__ to call). I propose that for mapping operations, Python must to change to use the "{}" operator: c = C() print c[0] # Print item with ordinal 0 print c{0} # Print item with key 0 According to my earlier proposal, "obj[item]" would be shorthand for "obj as Sequence.__getattr__(item)" and "obj{item}" would be shorthand for "obj as Mapper.__getattr__(item)". ==== I would appreciate your comments. /J/ -- James Dawes-Cooke: James.D-C@dial.pipex.com From jim.fulton@Digicool.com Mon Nov 23 18:18:15 1998 From: jim.fulton@Digicool.com (Jim Fulton) Date: Mon, 23 Nov 1998 13:18:15 -0500 Subject: [Types-sig] Re: PRE-PROPOSAL: Python Interfaces References: <3642E80C.4A997D94@digicool.com> <364c579e.516129164@news.triode.net.au> <72soaa$j2e$1@Starbase.NeoSoft.COM> <36528f91.923668414@news.triode.net.au> <72uc6e$kio$1@Starbase.NeoSoft.COM> Message-ID: <3659A6E7.D89482F4@digicool.com> This discussion should move to the types-sig, types-sig@python.org, http://www.python.org/sigs/types-sig/. I am cross posting to the types sig. Replies to this message should be posted to types-sig@python.org and *not* to comp.lang.python. Uche Ogbuji wrote (in regards to the developers's day session on interfaces): > > Actually, I didn't agree with a word that was said at that session, > and I am mortally terrified that after some initial resistance, Guido > appeared to be acquiescent to their suggestions. I tried (rather > incoherently) to express my disagreement, but I was mostly > misunderstood, of my own fault. > > I _much_ prefer the direction towards which Mr. Skaller is tending. > I don't know how many times it has to be re-proven that static type > systems are brittle, and rarely solve the problems they are supposed > to. I would rather that the latest demonstration were not in my > favorite language. Are you refering to the "Interfaces" session or the "Optional Static Types" session? Adoption of the ideas presented in the Interfaces session provides benefits without and doesn't require "Optional Static Types". Jim -- Jim Fulton mailto:jim@digicool.com Technical Director (888) 344-4332 Python Powered! Digital Creations http://www.digicool.com http://www.python.org Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats. From jim.fulton@Digicool.com Mon Nov 23 18:31:59 1998 From: jim.fulton@Digicool.com (Jim Fulton) Date: Mon, 23 Nov 1998 13:31:59 -0500 Subject: [Types-sig] "Interfaces" Jargon Message-ID: <3659AA1F.D220006E@digicool.com> I'd like to get moving on resolving jargon issues wrt to what I like to call "interfaces". I proposed the term "interface" for a thing that describes the external interface to and behavior of an object (or class instances). Alternative terms include: "specification", "behavior", "behavioral specification", "protocol", "type", "contract" ... I also proposed the term "implements" as in: "object X implements the Spam interface". An alternative term might be "conforms to". Someone suggested "indicates". "Interface" and "implements" are the terms used by Java. While there are differences between the interfaces idea that I've proposed and Java interfaces, I think that the similarities are sufficient enough to use the same terms. OTOH, I really don't give a %$#@ what jargon we use. I'd like to settle on *some* jargon so we can move forward with the Scarecrow proposal. :) At the "Python Interfaces" session ad Developer's Day '98, I took a straw pole and most respondents seemed to favor the terms "interfaces" and "implements", including Guido, Jim H., and myself, FWIW. If anyone *really* has a hard time using "interfaces" and "implements" or if you think a complelling case can be made against these or for others, then please speak up now. Jim -- Jim Fulton mailto:jim@digicool.com Technical Director (888) 344-4332 Python Powered! Digital Creations http://www.digicool.com http://www.python.org Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats. From guido@CNRI.Reston.VA.US Mon Nov 23 18:45:36 1998 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Mon, 23 Nov 1998 13:45:36 -0500 Subject: [Types-sig] "Interfaces" Jargon In-Reply-To: Your message of "Mon, 23 Nov 1998 13:31:59 EST." <3659AA1F.D220006E@digicool.com> References: <3659AA1F.D220006E@digicool.com> Message-ID: <199811231845.NAA21484@eric.cnri.reston.va.us> > If anyone *really* has a hard time using "interfaces" and "implements" > or if you think a complelling case can be made against these or for > others, then please speak up now. Oh my God! Now the rest of this year we'll be bickering over nothing but terminology issues. The problem is, in my experience the number of people who have (and want to express!) an opinion on what something is called is usually at least ten times the number of people who actually contribute over what it *is*. So, let's ignore terminology posts from people who haven't contributed to the meat of this thread before; once we've got a good understanding of the issues we can stop to think again about terminology. (0.5 :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From evan@tokenexchange.com Mon Nov 23 18:54:48 1998 From: evan@tokenexchange.com (Evan Simpson) Date: Mon, 23 Nov 1998 12:54:48 -0600 Subject: [Types-sig] Meta-classes discussion starter Message-ID: <16e101be1712$c0a79190$2d232cd1@linwin.token.hapenney.com> As much interest as I have in the whole issue of "interfaces / protocols / behaviors / stereotypes" (which I *will* be contributing to), I have more to say right now about the business of meta-classes and class methods. Thus, I'd like to kick off this thread with a proposal: ########## the abstract ########## It seems to me that there is really only one thing which distinguishes a meta-class from a class, if we want to treat everything as an object; A meta-class instantiates a class instead of an instance. This differs from an object which is simply capable of *manufacturing* a class object in that we want a class which sort-of-inherits its attributes from another class without actually being a sub-class. Thus instancing extends orthogonally from the class DAG, and attribute inheritance runs along the instance chain back to the class DAG and then up it in the usual depth-first search. If a meta-class produced instance classes which were just copies of itself, that would be only mildly interesting. Much more interesting would be the ability to selectively bind methods to the class instance to create class methods. In fact, I suggest that a meta-class be *defined* as a class which is capable of selective method binding. A class which binds all of its methods creates ordinary instances rather than classes. Now we need some way to declare how many levels of instantiation to defer the binding of each class method, and a way to keep track of this declaration in the actual class object. I suggest that some syntax be provided which allows a number of levels to be specified, with ordinary method definitions defaulting to normal binding behavior. An attribute of the class would be set to a dictionary of deferred-binding methods (DBM) levels by level number, each holding a dictionary of DBMs by name. IMPORTANT: Only normal binding methods would be set as attributes of the class, and only DBMs would be stored in the level dictionary. Thus ordinary classes would not require the level dictionary, and methods which override meta-class methods (esp. "__init__") could be defined without any name mangling. ########## the concrete ########## #(this is a proposed syntax, but not one I'm wedded to - I just want to make all the abstract stuff above a mite clearer): class MMC: def __init__(self, x): # Initialize MC self.x = x def[1] __init__(self, y): # Initialize C self.y = y def[2] __init__(self, z): # Initialize c self.z = z def printx(self) print x def[1] printy(self) print y def[2] printz(self) print z def[0] printxyz(self): # Bind to eventual non-class instances print x, y, z MC=MMC(1) C=MC(2) c=C(3) # printxyz is a bound method of c, and an unbound method of C # printx is a bound method of MC, and an unbound method of MMC # printy is a bound method of C, an unbound method of MC, and not defined in MMC # printz and printxyz are bound methods of c, unbound methods of C, and undefined in MC and MMC. # NOTE: c.printx, C.printx, and MC.printx all refer to the same bound method of MC, and all print the value bound to MC's "x" regardless of whether it has a different binding in C or c. (That's why we call it a class method :-) class MMMD(MMC): def[3] __init__(self): # Add another level of meta-classness to MMC pass # printxyz is bound to MMMD(x)(y)(z)(), not MMMD(x)(y)(z), because we specified a floating (zero) deferral level. printz, on the other hand, is bound to MMMD(x)(y)(z). class E: def a(self): pass def[0] b(self): pass # E is a normal class, since there are no fixed deferral levels to make the 'floating' method b bind differently, but it does keep track of the 'floatingness' of b, since... class MF(E): def[1] c(self): pass # MF is a meta-class with one unbound method "a", MF() is a class with class method "a" and unbound methods "b" and "c", and MF()() is an object with bound methods "b" and "c", and access to class method "a". # We can get MMC's __init__ at any level simply by writing MMC.__init__. To get C's __init__ from within MMC, we have to write: MMC.__def__[2]['__init__'] # which could be sugared to MMC[2].__init__ if we wanted. Note that "MMC[2].__init__ is MC[1].__init__ is C.__init__" but that there is no "C[0].__init__" since zero is for floating methods. Unless floating methods prove useful, this may be sufficient motivation to leave them out. ########## more dicussion ########## The fact that a class either lacks a __def__, has an empty one, or has one containing only zero-level, is what causes it to produce instances rather than classes. In particular, class C above has only a zero-level method in its __def__ attribute. A consequence of this is that if __def__ is mutable, the meta-ness of a class is a *dynamic run-time attribute*! In particular, we could make an instance of a class, then turn it into a meta-class, instantiate that as a class, then transmogrify the meta-class back into a class. Eeek. From papresco@technologist.com Mon Nov 23 16:59:09 1998 From: papresco@technologist.com (Paul Prescod) Date: Mon, 23 Nov 1998 10:59:09 -0600 Subject: [Types-sig] Re: PRE-PROPOSAL: Python Interfaces References: <1300476788-19595006@hypernet.com> <365700BD.FE8B1776@technologist.com> Message-ID: <3659945D.72B26EFE@technologist.com> Uche Ogbuji wrote: > > It would appear from the above that every language ever > written has "interface" (there are usually contraints to the use of > constructs in every language). What's special about Python in that > regard? Python is special in that it has a) an idea that objects of different types can be interchangable for each other, even if they do not share a common base class (other than PyObject). and b) that the categories of interchangability can be named. Today we name them "protocols." Tomorrow we will name them "interfaces." I don't think that the name change implies a change to the underlying idea. (does it?) Smalltalk doesn't allowing like that. C++ STL has an implicit (not language-enforced) concept that is similar (i.e. random-access iterator). Java's interfaces also capture some part of the problem. All I am asking as the next step is that it should be possible a) to define new categories of interchangability b) to ask an object what its categories of interchangability ("interfaces", or "protocols") are. What exactly do you dislike about being able to ask that question? Actually, I've been more annoyed about the lack of ability to ask about interfaces in terms of introspection instead of safety. For instance it annoys me that the Python Object Browser can browse the built-in types, and even C extension types, but not my Python-created sequence and mapping types. Argh. Now once the introspection feature is installed, I admit that I will likely use it to do runtime checks in cases where I think the caller is likely to make a mistake. If I'm controlling a nuclear plant, I want an explicit statement from the caller to the ShutDown() method that they understand that the first argument is some form of ShutDownController object and not some form of MeltDownController object. > If you think anyone is arguing against a method for > communicating and enforcing such constraints, I haven't heard that > argument. I have misgivings specifically against what Java calls > "interfaces", and the sort of derivative idea I heard at the > Developer Day. I'm a little lost, because I'm not clear what exactly you dislike about the strawperson proposal. > If I could compile this out of Python at installation, I don't > suppose I would complain. I don't have anything against others > getting language features that they desire, but I would rather not > carry the baggage of such features in the interpreter if I never > intended to use them. Let me see if I understand you right: you would be bothered by the mere fact that the compiler caught some issues that were *explicitly errors* at compile time instead of runtime: a="abcd" for i in range( a ): ... Would the overhead of checking this at compile time bother you? Even if it increased performance at runtime? And would it also bother you if you were allowed to provide some hints so that the compiler could recognize when your myrange() function was wrongly applied to a myInternationalizedString? Paul Prescod - http://itrc.uwaterloo.ca/~papresco A traceback is better than an unadorned exception. An exception is better than a naked core dump. A core dump is better than silent failure. Success beats all. Is Python the next step up your language ladder? From GoldenH@littoncorp.com Mon Nov 23 19:59:15 1998 From: GoldenH@littoncorp.com (Golden, Howard) Date: Mon, 23 Nov 1998 11:59:15 -0800 Subject: [Types-sig] Request and suggestions for proceeding Message-ID: I am closer to a Python newbie than a language lawyer, but I hope you will consider my request and suggestions for how to proceed with this SIG's task. * I believe that there may be a fundamental difference of goals among the interested parties, so I suggest making the goals explicit. * I believe that discussions can become overly intricate when dealing with unstated hypothetical situations. Therefore, I suggest stating concrete problems to solve. Let me take a whack at this: ---- What are my goals? ---- [From the Executive Summary:] "Python is an interpreted, interactive, object-oriented programming language. It is often compared to Tcl, Perl, Scheme or Java. "Python combines remarkable power with very clear syntax. It has modules, classes, exceptions, very high level dynamic data types, and dynamic typing...." [My position:] I want to develop large systems in Python, rather than C/C++ or Visual Basic, and I want to be able to reuse my code as well as others. I value the interactive nature of Python's implementation, but is this an essential feature of the language? Similarly, being interpreted and dynamically typed have caused certain essential design decisions. These may conflict with other desirable attributes. Specifically, dynamic typing is great for scripting but not for programming in the large. Therefore, I state my concerns: * I value the "very high level" list and dictionary data types. * I value the object oriented features. * I want to have optional static data typing and optionally required variable declaration, especially for type and "typo" safety, but also for performance. * I would prefer the (optional?) replacement of current namespace rules (local vs. global allows global variable access but only local variable modification), with something more like C/C++ (i.e., static rather than dynamic). * I want to use generic programming (especially being able to incorporate others' generic routines). However, I am unsure how this impacts the other features. From my perspective, there are many lurking rattlesnakes in the current mixture of language and implementation, especially for someone coming to Python from C/C++. ----What problems do I want to solve?---- For me, as stated above, I want to be able to use Python, rather than C/C++ or Visual Basic, to develop large business applications, especially using GUIs and databases. Basically, anything that Microsoft currently trots out Visual C++ for, I'd like to do in Python. (If you'll permit me to say it, I believe Python is the right way to do what Visual Basic was designed for.) I value the large system programming features more than the scripting and glue features. ----Summary---- What are the goals for Python? What problems are we trying to solve? Once these are clearer, it will be easier to choose which language and implementation alternatives to pursue. I have stated my goals. I recognize these may not be GvR's goals, or the goals of the Python community. If meeting my goals is in conflict with some goals you may have, I hope you will specifically describe your goals, so that I can try to understand them better. Respectfully, Howard B. Golden Software developer Corporate Office Litton Industries, Inc. Woodland Hills, California From gmcm@hypernet.com Mon Nov 23 20:52:29 1998 From: gmcm@hypernet.com (Gordon McMillan) Date: Mon, 23 Nov 1998 15:52:29 -0500 Subject: [Types-sig] "Interfaces" Jargon In-Reply-To: <3659AA1F.D220006E@digicool.com> Message-ID: <1300285618-4349014@hypernet.com> > I'd like to get moving on resolving jargon issues wrt to what I like > to call "interfaces". One firm vote for I don't particularly care, "interface" / "implements" is fine by me. - Gordon From JOrendorff@ixl.com Mon Nov 23 23:06:35 1998 From: JOrendorff@ixl.com (JOrendorff@ixl.com) Date: Mon, 23 Nov 1998 15:06:35 -0800 Subject: [Types-sig] Re: PRE-PROPOSAL: Python Interfaces References: <1300476788-19595006@hypernet.com> <365700BD.FE8B1776@technologist.com> <3659945D.72B26EFE@technologist.com> Message-ID: <3659EA7B.26675B7F@ixl.com> > Let me see if I understand you right: you would be bothered by the > mere fact that the compiler caught some issues that were *explicitly > errors* at compile time instead of runtime: > > a="abcd" > for i in range( a ): Hmmm... I just think the devil's in the details. ;-) In the spirit of Mr. Golden, here are my goals: 0. I use Python to goof off. So I want Python to be more fun to use, which means it catches more errors, sooner, and gives more helpful error messages, without requiring me to do any (er, much) extra work. To that end: 1. I'd like optional static typing to appear in Python 1.6 if possible. 2. I want the road to be clear for type inference, which is kind of the Holy Grail of goal #0. We shouldn't do anything now that type inference will do better a year from now. 3. I definitely want any features developed for 1.6 to be applicable to the 1.6 standard library, if possible. (Without breaking any working Python 1.x code, of course.) 4. Like Mr. Golden, I think all names should be bound as early as possible. But I think Python 1.6 must continue doing late binding of globals and builtins. I am especially interested in ideas for accomplishing #3. #4 suggests Python 1.6 won't catch 'range("abcd")' at compile-time. Are there any specific static typing proposals that have gotten at least a contemplative silence from Guido? :-) From what I've read, none yet has fared too well. -- Jason From jim.fulton@Digicool.com Mon Nov 23 23:13:30 1998 From: jim.fulton@Digicool.com (Jim Fulton) Date: Mon, 23 Nov 1998 18:13:30 -0500 Subject: [Types-sig] Re: PRE-PROPOSAL: Python Interfaces References: <3656BB11.641FE3ED@technologist.com> <1300476788-19595006@hypernet.com> Message-ID: <3659EC1A.8EDD4676@digicool.com> This discussion should move to the types-sig, types-sig@python.org, http://www.python.org/sigs/types-sig/. I am cross posting to the types sig. Replies to this message should be posted to types-sig@python.org and *not* to comp.lang.python. Gordon McMillan wrote: > > Paul Prescod writes: > ... > > By default, Python will be > > dynamic and sections can be made static in order to increase their > > safety or performance. If that can avoid the need to create a C > > extension even half of the time, it will have paid for itself. I > > would rather write my "inner loops" in statically typed Python than > > in C or Java. > > I wholeheartedly agree with Paul here. But I notice that his real > concern (like mine) appears to be "performance". I know that Roger > (and, I suspect, Jim F) are more concerned with "safety". I'd like to draw a distinction between the interface proposal and the "Optional Static Typing" proposal. I think interfaces would provide alot of benefits even if there were no argument/ variable type annotations. Personally, I consider myself, uh, unenthusiastically open minded about "static typing". > My suspicion is that, after a brief honeymoon, these two concerns > will immediately start tugging in opposite directions. (Hmmm... Jim > "The Bear" Fulton vs Jim "The Bee" Hugunin; could be > entertaining...). I think they are two different problems and I expect different solutions. They need not be competing. If they are, then somethings wrong, IMO. Jim -- Jim Fulton mailto:jim@digicool.com Technical Director (888) 344-4332 Python Powered! Digital Creations http://www.digicool.com http://www.python.org Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats. From jim.fulton@Digicool.com Tue Nov 24 00:22:43 1998 From: jim.fulton@Digicool.com (Jim Fulton) Date: Mon, 23 Nov 1998 19:22:43 -0500 Subject: [Types-sig] Re: PRE-PROPOSAL: Python Interfaces References: <3656BB11.641FE3ED@technologist.com> <1300476788-19595006@hypernet.com> Message-ID: <3659FC53.72D9FA32@digicool.com> This discussion should move to the types-sig, types-sig@python.org, http://www.python.org/sigs/types-sig/. I am cross posting to the types sig. Replies to this message should be posted to types-sig@python.org and *not* to comp.lang.python. Gordon McMillan wrote: > > Paul Prescod writes: > ... > > By default, Python will be > > dynamic and sections can be made static in order to increase their > > safety or performance. If that can avoid the need to create a C > > extension even half of the time, it will have paid for itself. I > > would rather write my "inner loops" in statically typed Python than > > in C or Java. > > I wholeheartedly agree with Paul here. But I notice that his real > concern (like mine) appears to be "performance". I know that Roger > (and, I suspect, Jim F) are more concerned with "safety". I'd like to draw a distinction between the interface proposal and the "Optional Static Typing" proposal. I think interfaces would provide alot of benefits even if there were no argument/ variable type annotations. Personally, I consider myself, uh, unenthusiastically open minded about "static typing". > My suspicion is that, after a brief honeymoon, these two concerns > will immediately start tugging in opposite directions. (Hmmm... Jim > "The Bear" Fulton vs Jim "The Bee" Hugunin; could be > entertaining...). I think they are two different problems and I expect different solutions. They need not be competing. If they are, then somethings wrong, IMO. Jim -- Jim Fulton mailto:jim@digicool.com Technical Director (888) 344-4332 Python Powered! Digital Creations http://www.digicool.com http://www.python.org Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats. From jim.fulton@Digicool.com Tue Nov 24 00:32:11 1998 From: jim.fulton@Digicool.com (Jim Fulton) Date: Mon, 23 Nov 1998 19:32:11 -0500 Subject: [Types-sig] Re: PRE-PROPOSAL: Python Interfaces References: <365700BD.FE8B1776@technologist.com> <1300449328-21246608@hypernet.com> Message-ID: <3659FE8A.3DAA175C@digicool.com> This discussion should move to the types-sig, types-sig@python.org, http://www.python.org/sigs/types-sig/. I am cross posting to the types sig. Replies to this message should be posted to types-sig@python.org and *not* to comp.lang.python. Gordon McMillan wrote: > > I don't object, but I worry. For example, the "sequence protocol" is > a very fuzzy thing. Yes it is. It should be factored. > If you start with the abstract sequence protocol (as defined by the C > API), only List is an implementation of which B. Meyer would approve, > (SetItem is in the abstract interface, for example). Yup. > OK, so lets pretend that's cleaned up. Now we're getting methods on > List so it can be treated like a Stack. Stack is certainly not > another name for List, nor for Sequence (whatever that means). It's > another protocol, right? Yes. > But it's one that is automatically > implemented if you implement the List protocol. Right. If you bothered to define "List" protocol as the methods supported by list, you might very well make stack a base interface. > How about FileInput? That's a sequenceish protocol. A read-only, > forward-only, sequential-only sequence. How do we (sensibly and > comprehensibly) capture the fact that most sequenceish protocols will > suffice where a FileInput protocol is required, with the exception of > those stack-ish style sequences? I don't know what FileInput is. > In fact (as Barry was supposed to point out at the conference for me, > harrumph, harrumph), the set of all interfaces / protocols is the set > of all subsets of the set of all signatures - without taking behavior > into account. That is, the set of all interfaces has a cardinality > that I don't particularly want to contemplate. But one would not bother to enumerate all possible interfaces. > At the moment "sequence protocol" means 2 different things. It means > Py_SequenceCheck(o) == TRUE (which I hope the type / class > unification fixes), No. This is a purely implementation check. It doesn't, for example, capture instances that implement the sequence protocol. > and it means that if you squint a little and wave > your hands alot, this thing could be considered a sequence. My worry > is that if you attempt to make the latter definition more precise, > you in fact increase complexity. Interfaces give you the ability to express more. In a sense this adds complexity, but it also lends order to existing complexity (e.g., as you point out, Tuples are sequences, sort of) Jim -- Jim Fulton mailto:jim@digicool.com Technical Director (888) 344-4332 Python Powered! Digital Creations http://www.digicool.com http://www.python.org Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats. From jim.fulton@Digicool.com Tue Nov 24 00:41:39 1998 From: jim.fulton@Digicool.com (Jim Fulton) Date: Mon, 23 Nov 1998 19:41:39 -0500 Subject: [Types-sig] Re: PRE-PROPOSAL: Python Interfaces References: <3642E80C.4A997D94@digicool.com> <3oE02.5$Hg.47@news-dal.corridex.com> <72gcjo$je1$1@thetenth.astat.de> <364c579e.516129164@news.triode.net.au> <72soaa$j2e$1@Starbase.NeoSoft.COM> <36528f91.923668414@news.triode.net.au> <3657123A.66D48330@technologist.com> Message-ID: <365A00C3.BBC57A86@digicool.com> This discussion should move to the types-sig, types-sig@python.org, http://www.python.org/sigs/types-sig/. I am cross posting to the types sig. Replies to this message should be posted to types-sig@python.org and *not* to comp.lang.python. Paul Prescod wrote: > > This is almost identical to what we decided to implement at the > conference. The only catch is that __class_protocols__ is called > __interfaces__ and it is a list of interface objects. An interface object > looks a lot like a class (because we are using existing Python syntax > features) but it is conceptually a class with no implementation. Actually, in my mind (and current implementation) it's actual and conceptual connection to a class is pretty tenuous. Asside from being created with a class statement, it doesn't look much like a class. Interfaces are not classes and can't be subclassed with classes. Jim -- Jim Fulton mailto:jim@digicool.com Technical Director (888) 344-4332 Python Powered! Digital Creations http://www.digicool.com http://www.python.org Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats. From jim.fulton@Digicool.com Tue Nov 24 01:04:52 1998 From: jim.fulton@Digicool.com (Jim Fulton) Date: Mon, 23 Nov 1998 20:04:52 -0500 Subject: [Types-sig] Scarecrow Proposal References: <3657ECA1.6B2BD52@lemburg.com> Message-ID: <365A0634.9D7B079F@digicool.com> "M.-A. Lemburg" wrote: > > I've just read the Scarecrow Proposal and have a few questions > (sorry, if these have already been answered at the conference... > I haven't been there): > > · Why is it important that classes and interfaces are different ? Did you read the PRE-PROPOSAL, on which this is based? http://x14.dejanews.com/getdoc.xp?AN=409004574&CONTEXT=911406505.481755173 > Is this just to make sure you can't mix classes and interfaces > in __bases__ ? No, it is to: - Reinforce the separation between implementation and interface, and - To avoid requiring that subclasses must implement all the interfaces of base classes. > IMHO, we should use as much as possible from the > existing features, so I think building the interface hierarchy > with plain Python classes Why do you care that plain Python classes are used. > and then sticking them into the class > hierarchy as __implements__ (should probably be named __interfaces__; > a builtin function implements() could take care of the actual > lookup in whatever is suitable). Yes. > If there are strong reasons for making the two different in the > sense that type(class) is not type(interface), then I'd opt for > simply cloning the class implementation by creating a second > type object that is simply a copy of the class' one. This is not necessary. It's easy to use the meta-class hook to have class statements create Interface instances that are not classes. > · Would this proposal get us closer to static typing ? If we ever do static typing, then I would hope that the static typing would use interfaces. Other than that, I don't know. > I'm thinking of a way to declare a named object ABC as having > an interface XYZ (e.g. a function could be assigned an interface > stating that the first argument must be an integer). > > Since the interface hierarchy is something that is built dynamically, > the compiler would have to be given two things: an already executable > script INT that builds the interface hierarchy and the source script SRC > referring to this hierarchy. It could then optimize e.g. the function > calls > to abc knowing that the first argument is an integer and also add > checks at C level to ensure this (a PyInt_Check() is a whole lot > faster than the same check via type() in Python). I really don't think that the interface mechanism that I propose is going to be very useful for optimization. For example, knowing that an object conforms to some "integer" interface should not tell you that it has a particular implementation, so, for example, you can't assume that it's of type PyInt_Type. > A general syntax would have to be introduced to accomplish this > (which would have to go into Python 2.0 I presume), e.g. > > def ABC(a,b,c) implements TypedArguments(Integer,Any,Any): > a = a + 1 > return a, b+c > > where TypedArguments is a helper function defined in the INT script > which creates an interface for the given argument combination (Integer > and Any would also have to be defined in INT and the compiler could > call methods on these to find out how to optimize them). > > Note: The helper function would have to be called at *compile time*. > > Also note: The code INT could have been imported by the compiler > as module prior to compilation, e.g. if there were a standard > hierarchy available no extra fuzz would be implied for the > compilation step. Note that assigning types to variables is a different issue from the interface issue. > · In any case, is the interface proposal going to effect any > existing code ? > > I would like to see it implemented as additional useful feature, > but in a way that is orthogonal to the existing implementation. > Especially: no performance hit if you don't use it. The scarecrow proposal adds no performance hit. > · Should interfaces also allow changing the method invokation > behaviour ? > > E.g. pre- and post-conditions could be defined in the interface > rather than in a meta-class. When the compiler sees a method > ABC_precondition in the interface it generates code to call > this function prior ABC. Same for ABC_postcondition. > > Do these Eiffel constructs belong into interfaces ? IMHO, they > do. This is a topic for discussion post-Scarecrow. Jim -- Jim Fulton mailto:jim@digicool.com Technical Director (888) 344-4332 Python Powered! Digital Creations http://www.digicool.com http://www.python.org Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats. From gmcm@hypernet.com Tue Nov 24 05:03:09 1998 From: gmcm@hypernet.com (Gordon McMillan) Date: Tue, 24 Nov 1998 00:03:09 -0500 Subject: [Types-sig] Request and suggestions for proceeding In-Reply-To: Message-ID: <1300256177-1197682@hypernet.com> Howard Golden makes a plea for explicit goals: > I want to develop large systems in Python, rather than C/C++ or > Visual Basic, and I want to be able to reuse my code as well as > others. Worthy goal. > I value the interactive nature of Python's implementation, but is > this an essential feature of the language? Yes. (IMO, of course). > Similarly, being > interpreted and dynamically typed have caused certain essential > design decisions. These may conflict with other desirable > attributes. Specifically, dynamic typing is great for scripting but > not for programming in the large. I don't quite buy that. First, I don't think anyone is considering removing dynamic typing. Some want to add optional static typing (and that's one of the topics for this SIG), but there's nothing "static" about the other 2 topics (interfaces and resolving the class / type divergence). Second, I know when I started with Python I felt the need for static typing. For me, at least, I think this was a carryover from 15 years of working with statically typed languages. Since then, my development style has changed dramatically. I am now inclined to "play" with the problem and try many different ways of factoring it (instead of working out sketches of a few, then picking one and sticking with it). Ultimately, I arrive at a much cleaner solution - with much smaller and simpler interdependencies. At least some of the time, these solutions would be harder to express if they were statically typed. > * I want to use generic programming (especially being able to > incorporate others' generic routines). However, I am unsure how > this impacts the other features. You're asking for SPython (Static Python). Generic programming is already a feature of dynamically typed languages. I, too, want SPython, but not as a replacement for Python. > I have stated my goals. I recognize these may not be GvR's goals, > or the goals of the Python community. If meeting my goals is in > conflict with some goals you may have, I hope you will specifically > describe your goals, so that I can try to understand them better. I use Python in 3 ways. As a prototyping tool (most of my clients insist on C++ or C), as a system development language, and as a scripting tool (both Unix style munging stuff, and some Windows style glue components together stuff). The Unix style munging is probably it's own category (which I might be willing to sacrifice), but the others heavily overlap. I'm interested in seeing Python extend its grip in all those areas. That means I'm interested in optional static typing if it gets me increased performance and safety, but not if it excessively limits my options. Much the same for interfaces - if it helps me express an abstract solution cleanly and understandably, I'm all for it, but not if I end up having to work around it. I'm unequivocal about the class / type topic - this is a real "gotcha" in Python. - Gordon From mal@lemburg.com Tue Nov 24 09:37:28 1998 From: mal@lemburg.com (M.-A. Lemburg) Date: Tue, 24 Nov 1998 10:37:28 +0100 Subject: [Types-sig] Scarecrow Proposal References: <3657ECA1.6B2BD52@lemburg.com> <365A0634.9D7B079F@digicool.com> Message-ID: <365A7E58.7111F703@lemburg.com> Jim Fulton wrote: > > "M.-A. Lemburg" wrote: > > > > I've just read the Scarecrow Proposal and have a few questions > > (sorry, if these have already been answered at the conference... > > I haven't been there): > > > > · Why is it important that classes and interfaces are different ? > > Did you read the PRE-PROPOSAL, on which this is based? > http://x14.dejanews.com/getdoc.xp?AN=409004574&CONTEXT=911406505.481755173 To be honest: I just flew over it at the time and then decided it was a good thing but not something that would interest me too much. After a while I started to see that this could be the solution to providing "static" typing in a very smart way (with interfaces being used as "type" tags providing all kinds of nifty details to possible optimizing engines -- which is a great advantage over just defining X to be an integer: the optimizer could query the type to find out the best and most suitable way to produce code for it; it might even be able to decide which algorithm to pick depending one the features). That got me interested again and so here I am ;-) The above question is not so much about strategy, it is about ease of implementation: interfaces could benefit from inheritance too. Since interfaces and classes belong to two worlds this would have the great benefit of reuse for interfaces too (e.g. to specify the Sequence protocol in all kinds of different flavors). > > Is this just to make sure you can't mix classes and interfaces > > in __bases__ ? > > No, it is to: > > - Reinforce the separation between implementation and > interface, and > > - To avoid requiring that subclasses must implement > all the interfaces of base classes. This is another thing I didn't understand in your proposal: why do interfaces have to actually define method signatures at all ? Since interfaces will be used as tags only this would confuse the hell out of newbies when they find that they can't call those methods. IMHO, interfaces ought to have information query methods, not "real" ones that are only given to describe which methods are expected to be implemented by a class conforming to an interface. I would put more emphasis on the tagging nature of interfaces rather than making them look like a C++ base class. The methods signature of the interface could well be stored using other means, e.g. by using a class defined only for this purpose: # Create a Sequence interface singleton class Sequence(Interface): # Class defining the signatures of the methods that # a Sequence conforming class must define class signature: def __getitem__(self,i): """ Get item number i from the sequence """ def check(self,Class): """ Check whether Class actually conforms to the signatures defined above. """ # Flags for the optimizer getitem_indices_are_integers = 1 Sequence = Sequence() # A class stating conformance to the Sequence interface: class MyList(UserList): __interface__ = (Sequence,) > > IMHO, we should use as much as possible from the > > existing features, so I think building the interface hierarchy > > with plain Python classes > > Why do you care that plain Python classes are used. Makes things simpler. Classes already provide most of the functionality needed, IMHO, and so why not simply reuse them ?! > > and then sticking them into the class > > hierarchy as __implements__ (should probably be named __interfaces__; > > a builtin function implements() could take care of the actual > > lookup in whatever is suitable). > > Yes. > > > If there are strong reasons for making the two different in the > > sense that type(class) is not type(interface), then I'd opt for > > simply cloning the class implementation by creating a second > > type object that is simply a copy of the class' one. > > This is not necessary. It's easy to use the meta-class hook > to have class statements create Interface instances that are > not classes. Ok. But why not use singletons as the one above ? > > · Would this proposal get us closer to static typing ? > > If we ever do static typing, then I would hope that the > static typing would use interfaces. Other than that, I don't know. > > > I'm thinking of a way to declare a named object ABC as having > > an interface XYZ (e.g. a function could be assigned an interface > > stating that the first argument must be an integer). > > > > Since the interface hierarchy is something that is built dynamically, > > the compiler would have to be given two things: an already executable > > script INT that builds the interface hierarchy and the source script SRC > > referring to this hierarchy. It could then optimize e.g. the function > > calls > > to abc knowing that the first argument is an integer and also add > > checks at C level to ensure this (a PyInt_Check() is a whole lot > > faster than the same check via type() in Python). > > I really don't think that the interface mechanism that I propose > is going to be very useful for optimization. For example, knowing > that an object conforms to some "integer" interface should not tell you > that it has a particular implementation, so, for example, you can't assume > that it's of type PyInt_Type. But I could define an Integer interface to mean: objects conforming to this interface will work fine with the builtin int() and the optimizer may convert them using int() to enable better ways of storing their values. (The object could also define a reverse method, e.g. reinit_from_int() to reverse that trick -- all well definable in an interface definition.) > > A general syntax would have to be introduced to accomplish this > > (which would have to go into Python 2.0 I presume), e.g. > > > > def ABC(a,b,c) implements TypedArguments(Integer,Any,Any): > > a = a + 1 > > return a, b+c > > > > where TypedArguments is a helper function defined in the INT script > > which creates an interface for the given argument combination (Integer > > and Any would also have to be defined in INT and the compiler could > > call methods on these to find out how to optimize them). > > > > Note: The helper function would have to be called at *compile time*. > > > > Also note: The code INT could have been imported by the compiler > > as module prior to compilation, e.g. if there were a standard > > hierarchy available no extra fuzz would be implied for the > > compilation step. > > Note that assigning types to variables is a different > issue from the interface issue. It is not that much different: see above. > > · In any case, is the interface proposal going to effect any > > existing code ? > > > > I would like to see it implemented as additional useful feature, > > but in a way that is orthogonal to the existing implementation. > > Especially: no performance hit if you don't use it. > > The scarecrow proposal adds no performance hit. Good :-) > > · Should interfaces also allow changing the method invokation > > behaviour ? > > > > E.g. pre- and post-conditions could be defined in the interface > > rather than in a meta-class. When the compiler sees a method > > ABC_precondition in the interface it generates code to call > > this function prior ABC. Same for ABC_postcondition. > > > > Do these Eiffel constructs belong into interfaces ? IMHO, they > > do. > > This is a topic for discussion post-Scarecrow. True. -- Marc-Andre Lemburg Y2000: 402 days left --------------------------------------------------------------------- : Python Pages >>> http://starship.skyport.net/~lemburg/ : --------------------------------------------------------- From jim@Digicool.com Tue Nov 24 13:03:09 1998 From: jim@Digicool.com (Jim Fulton) Date: Tue, 24 Nov 1998 13:03:09 +0000 Subject: [Types-sig] Scarecrow Proposal References: <3657ECA1.6B2BD52@lemburg.com> <365A0634.9D7B079F@digicool.com> <365A7E58.7111F703@lemburg.com> Message-ID: <365AAE8D.6D38911B@digicool.com> M.-A. Lemburg wrote: > > > To be honest: I just flew over it at the time and then decided it > was a good thing but not something that would interest me too much. > After a while I started to see that this could be the solution to > providing "static" typing in a very smart way (with interfaces being > used as "type" tags providing all kinds of nifty details to possible > optimizing engines Perhaps someone will prove me wrong, but I expect that optimization engines would want certain implementation information that would be absent from a purely behavioral specification. Hm. Maybe it would be worth consider additional meta objects (other than interfaces) that could be attributed to classes or objects to give optimization hints. > -- which is a great advantage over just defining > X to be an integer: the optimizer could query the type to find out > the best and most suitable way to produce code for it; it might even > be able to decide which algorithm to pick depending one the features). > That got me interested again and so here I am ;-) > > The above question is not so much about strategy, it is about > ease of implementation: interfaces could benefit from inheritance > too. With the Scarecrow proposal, interfaces get to use inheritence too. > Since interfaces and classes belong to two worlds this would > have the great benefit of reuse for interfaces too (e.g. to specify > the Sequence protocol in all kinds of different flavors). > > > > Is this just to make sure you can't mix classes and interfaces > > > in __bases__ ? > > > > No, it is to: > > > > - Reinforce the separation between implementation and > > interface, and > > > > - To avoid requiring that subclasses must implement > > all the interfaces of base classes. > > This is another thing I didn't understand in your proposal: > why do interfaces have to actually define method signatures > at all ? They don't, but most people want signatures to be included in interface specs. > Since interfaces will be used as tags only this would > confuse the hell out of newbies when they find that they can't > call those methods. If an interface provides a method signature, then an object that implements the interface is contractually bound to provide it too, so newbies and anyone else should feel free to call the object. Note that signatures will *not* be available as attributes of the interface. For example: class FooInterface(Interface): def spam(a, b): "blah blah blah :)" FooInterface.spam # raises an attribute error > IMHO, interfaces ought to have information query methods, not > "real" ones that are only given to describe which methods are > expected to be implemented by a class conforming to an interface. I agree. > I would put more emphasis on the tagging nature of interfaces > rather than making them look like a C++ base class. I couldn't agree more. > The methods signature of the interface could well be stored > using other means, e.g. by using a class defined only for this > purpose: > > # Create a Sequence interface singleton > class Sequence(Interface): > > # Class defining the signatures of the methods that > # a Sequence conforming class must define > class signature: > def __getitem__(self,i): > """ Get item number i from the sequence > """ > > def check(self,Class): > """ Check whether Class actually conforms > to the signatures defined above. > """ > > # Flags for the optimizer > getitem_indices_are_integers = 1 This isn't necessary, since Interfaces won't be classes. > Sequence = Sequence() This won't be necessary either. class FooInterface(Interface): ... creates an *interface*, not an interface class. I'll post me initial implementation later today. This should make things a little bit clearer. > # A class stating conformance to the Sequence interface: > class MyList(UserList): > > __interface__ = (Sequence,) > > > > IMHO, we should use as much as possible from the > > > existing features, so I think building the interface hierarchy > > > with plain Python classes > > > > Why do you care that plain Python classes are used. > > Makes things simpler. Classes already provide most of the > functionality needed, IMHO, and so why not simply reuse them ?! Actually, they get in the way. Using separate objects is really much simpler, as I think you'll see when I post me code. :) > > > and then sticking them into the class > > > hierarchy as __implements__ (should probably be named __interfaces__; > > > a builtin function implements() could take care of the actual > > > lookup in whatever is suitable). > > > > Yes. > > > > > If there are strong reasons for making the two different in the > > > sense that type(class) is not type(interface), then I'd opt for > > > simply cloning the class implementation by creating a second > > > type object that is simply a copy of the class' one. > > > > This is not necessary. It's easy to use the meta-class hook > > to have class statements create Interface instances that are > > not classes. > > Ok. But why not use singletons as the one above ? The meta-class hook is simpler. 8-o I was surprized myself. > > > · Would this proposal get us closer to static typing ? > > > > If we ever do static typing, then I would hope that the > > static typing would use interfaces. Other than that, I don't know. > > > > > I'm thinking of a way to declare a named object ABC as having > > > an interface XYZ (e.g. a function could be assigned an interface > > > stating that the first argument must be an integer). > > > > > > Since the interface hierarchy is something that is built dynamically, > > > the compiler would have to be given two things: an already executable > > > script INT that builds the interface hierarchy and the source script SRC > > > referring to this hierarchy. It could then optimize e.g. the function > > > calls > > > to abc knowing that the first argument is an integer and also add > > > checks at C level to ensure this (a PyInt_Check() is a whole lot > > > faster than the same check via type() in Python). > > > > I really don't think that the interface mechanism that I propose > > is going to be very useful for optimization. For example, knowing > > that an object conforms to some "integer" interface should not tell you > > that it has a particular implementation, so, for example, you can't assume > > that it's of type PyInt_Type. > > But I could define an Integer interface to mean: objects conforming > to this interface will work fine with the builtin int() and the > optimizer may convert them using int() to enable better ways of > storing their values. (The object could also define a reverse > method, e.g. reinit_from_int() to reverse that trick -- all well > definable in an interface definition.) Fair enough. The interface should probably say, if it can, what the possible range of values is. (e.g. can a 16-bit integer be used at the C level, must a python long be used, etc.) Jim -- Jim Fulton mailto:jim@digicool.com Technical Director (540) 371-6909 Python Powered! Digital Creations http://www.digicool.com http://www.python.org Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats. From gmcm@hypernet.com Tue Nov 24 14:42:25 1998 From: gmcm@hypernet.com (Gordon McMillan) Date: Tue, 24 Nov 1998 09:42:25 -0500 Subject: [Types-sig] Re: PRE-PROPOSAL: Python Interfaces In-Reply-To: <3659FE8A.3DAA175C@digicool.com> Message-ID: <1300221419-3288205@hypernet.com> [me] > > How about FileInput? That's a sequenceish protocol. A read-only, > > forward-only, sequential-only sequence. How do we (sensibly and > > comprehensibly) capture the fact that most sequenceish protocols will > > suffice where a FileInput protocol is required, with the exception of > > those stack-ish style sequences? [Jim Fulton] > I don't know what FileInput is. Well why didn't "read-only, forward-only, sequential-only sequence" define it completely ? I doubt it's much more helpful to know that the only sequence method it implements is __getitem__. Saying that's __getitem__(Int) would help a little, but I'm afraid that looking at the source (it's in the standard distribution) or asking on the newsgroup are the only ways of finding out what a FileInput really _is_. > > That is, the set of all interfaces has a cardinality > > that I don't particularly want to contemplate. > > But one would not bother to enumerate all possible interfaces. True. One would not even try to enumerate all possible signatures. The point is, though, that there's a conflict between precision and usability. If interfaces are too precise, they will multiply with abandon and become unmanageable. To prevent that, people will choose the "closest" interface. So the "requirer" will ask for more than is actually needed, and the "requiree" will have the choice of adding unneeded methods, wrapping unnecessarily, or lying. > > At the moment "sequence protocol" means 2 different things. It means > > Py_SequenceCheck(o) == TRUE (which I hope the type / class > > unification fixes), > > No. This is a purely implementation check. It doesn't, for > example, capture instances that implement the sequence protocol. That's why I said it needs to be fixed. > > and it means that if you squint a little and wave > > your hands alot, this thing could be considered a sequence. My worry > > is that if you attempt to make the latter definition more precise, > > you in fact increase complexity. > > Interfaces give you the ability to express more. In a sense this > adds complexity, but it also lends order to existing complexity > (e.g., as you point out, Tuples are sequences, sort of) They actually express less. But, hopefully, they express it much more concisely, and at a "reaonable" level of abstraction. They do not obviate the need for reading the source (see FileInput, above), but once you've done that they serve as mnemonics. Once you've been through it, you now either remember, or know where to look it up. every-silver-lining-has-a-cloud-ly y'rs - Gordon From jim@Digicool.com Tue Nov 24 15:34:54 1998 From: jim@Digicool.com (Jim Fulton) Date: Tue, 24 Nov 1998 15:34:54 +0000 Subject: [Types-sig] Re: PRE-PROPOSAL: Python Interfaces References: <3642E80C.4A997D94@digicool.com> <364b1070.432371637@news.triode.net.au> Message-ID: <365AD21E.F25A4EAA@digicool.com> Note that this discussion should move to the Types SIG: http://www.python.org/sigs/types-sig/. I am cross posting to types-sig@python.org. Replies should be made to types-sig and omit comp.lang.python. John (Max) Skaller wrote: > > On Fri, 06 Nov 1998 12:14:04 +0000, Jim Fulton wrote: (meta comment snipped ;) > Skip to the end for positive considerations! > > >PROPOSED APPROACH > > > > At a minimum, I'd like to get some sort of agreement on a basic > > high-level approach by the end of the Developer Day interfaces > > session. > > > > Here is my proposal: > > > > - There should be a way of defining interfaces in Python. > > Seems fair enough. > > > - An interface describes abstract behavior. > > I have no idea what that really means.... This is intentionally vague. At minimum it is just a tag from a hierarchical tag space. Where we take it beyond that is a matter for discussion. > > - Interfaces are arranged in a hierarchy. Implementation of a > > sub-interface implies implementation of base interfaces. > > NO! This is vastly too restrictive: this case is almost useless. Why is this too restrictive. This is merely a tool to organize interfaces. This approach has used by many systems, so I have trouble buying that it is useless. > > - Interfaces will almost certainly not be *both* complete > > and formal. > > Why not? It can be done. Indeed, it has been. I did not say "can not", I said "will not". I have a little experience with formal languages, mostly Z, and know that one can built nifty formal behavioral models, but such formal models would be of little practical use to Python programmers and of even less use to the Python run-time. > >For example, interfaces might provide syntactical > > information (signatures) formally, but may provide behavioral > > information informally, as documentation strings. > > No need. Any abstraction can be defined completely, and formally. > If less is specified .. it is a weaker (less specific) abstraction. It can be, but it won't be. Very few Python programmers would have either the means or inclination. > > - There should be a way for a an object to assert that it > > implements one or more interfaces. > > I suspect this is wrong as a generalisation, although it may > be a useful special case. To give a specific example, in the function > call: > > f(a,b,c) > > it is not enough that a, b, and c each has a particular interface. > They must also stand in some relation (perhaps asserted by a precondition). It depends on how complete you are being. In any case, preconditions on a, b, and c can be simpler if they can test for certain interface assertions. > > The mechanism for asserting > > conformance to interfaces is independent of an object's class. > > Agree. > > > - There should be a way for a class to assert the (default) > > interfaces that are implemented by it's instances. This > > mechanism is independent of the classes implementation. In > > particular, this mechanism does not use class inheritance. > > Yes. But you must understand that it is _not_ objects that > have behaviours, and it is _not_ objects that have interfaces. > It is _functions_ that have behaviour and collections of functions > which must be related by semantic rules. > > It is fair to provide _convenience_ for functions collected > as methods of objects .. but this is only a special case. This is where we probably disagree is some fundamental (religious ;) way. I believe that objects should be the primary viewpoint for reasoning about programs. > > - The language may (is likely to) partially check assertions. > > The 'checking' side of specification is far less interesting > than the constructive side! > > I want to give an example. A 'class' is an example. > It is a mechanism to _construct_ multiple objects and can be > leveraged to ensure they all obey some protocol -- because they > all get the same set of methods. I don't mean to imply such obedience > follows in all cases, but that with well written code use of classes > can aid in verification: the construction is not strong enough for a guarrantee > without programmer effort and reader analysis, but it helps a lot. > > So I'd lke to suggest that 'checking' interfaces is missing > the most important application of your abstract goals: the very best > checks are those that are not necessary! That is, ones that are > assured _by construction_. > > > - There should be efficient tools to test: > > > > - Whether an object implements an interface, > > Objects do not really implement interfaces. > Interfaces are implemented by functions. Objects implement object interfaces. Object interfaces include method interfaces, as well as abstract state models that model dynamic relations between method calls. > > - Whether instances of a class (that do not make their own > > interface assertions) implement an interface, and > > > > - Whether an interface is a sub-interface of some other > > interface. > > This is the wrong terminology. Let's not get confused > with OO hype here There is no hierarchy. That is not > how it works. Interfaces (protocols) are related by categories. > Just to be sure you underdstand: Python classes _already_ > transcend any notion of heirachy. Python supports > multiple inheritance! You would have to move from a tree > model to DAG just to account for python classes! Of course, the interface "hierarchy" is really a DAG, just as the class "hierarchy" is really a DAG. > In the end, there is a complete theory, > in which a DAG is again only a trivial example. > > Category theory provides a complete theory of what you want. > Consider a collection of nodes with a single pointer. Perhaps they > form a tree. The problem is that I don't know what category theory is, and the myriad posts you've made haven't shed much light on the subject for me. Some material at the end of your post has made it a little clearer. I doubt that anyone else reading this thread who didn't already know what this was is much clearer on it now. The beauty of classification systems based on DAGs, which are, for the sake of simplicity, typically called hierarchies, is that they are very easily understood, and therefore useful. > Is there an object which 'is a' tree?? Yes. > No. No single object has 'tree protocol' or 'tree interface'. Depends on how you choose to define the interface. > Is it the collection of objects which forms a tree?? I might, and typically would, provide an object that manages the nodes and provides some assistance for navigating to nodes. > NO NO NO. It is the _pointers_ which form the tree! > Isn't that obvious, now I've said it ?? Your view is way to low-level for me. The beauty of OO is that it provides a system for building high-level abstractions that let you see the tree as a whole, if that is useful for the problem you are trying to solve. > OBJECTS ARE POINTS. THEY HAVE NO STRUCTURE. You are obviously using the term "object" in a way that is very different from the way that I use it. To me, objects are all about providing structure to data and algorithms. > They can't have structure -- they don't have enough 'dimension'. > You need _arrows_ to build structures. > > > - Jargon > > > > I like the term "interface", but I'd be happy to use other > > terminology. A number of folks prefer the term "protocol". > > Let's agree on a term and stick to it. > > To me there is a distinction. A protocol is a dynamic thing. > It involves not only static properties like having a set of methods > with certain signatures, but dynamic ones like: "if you call > method A, and then you call method B then ..." Calling those > methods in that order is a matter of protocol. But this is only > my personal taste :-) I agree with you that we are talking about more than signatures. This was one of the main points I was trying to make. Even if you can (or choose to) only express part of the behavior in the language, programmers need to be aware that there is more involved. This is why I think that the tagging nature of the scarecrow proposal is far more important than details like signatures. > > - What's in an interface? > > > > I've been intentionally vague about what's in an interface. Most > > people would expect interfaces to, at a minimum, include > > signatures. Frankly, I think interfaces would be valuable even if > > they only included hierarchy information (base interfaces) and doc > > strings. > > NO! Both ideas are wrong. Neither piece of information > is really important. What is important is to provide the _correct_ > tools for a general specification. > > So lets analyse the two ideas. First, method signatures > in static languages specify the codomain (return type) and > a superset of the domain: a cartesian product of the parameter > types. The actual domain is specified by refinement: usually > using exception specifications (C++) or preconditions (Eiffel). > > But neither mechanism is really enough. I agree. BTW, the signatures need not have "type" information at all. For example: def f(a,b,c=None): ... has a signature along the lines: accepts three positional arguments, a, b, and c, where c may be omitted. > But specifying > a set of categories which the method is an arrow of IS > a complete specification. And it doesn't involving naming ANY types > at all because it only involves laws of composition (chaining) > of methods. From what little I've seen, I can't see how to use it to form useful specifications. I'm sure if I worked hard enough at it I could, but most people don't want to work that hard. > This is absolutely mandatory for genericity. > I'm sure you will understand if I explain this by analogy to > information hiding: it is NOT enough for an interface specification > to elide implementation details to be abstract, it must not > mention any specific types either. "Type" is too loaded a word. What do you mean by "type"? Do you mean a Python type, which in reality is another kind of class? Or do you mean abstract data type? > They're really implementation details too! This discussion gets messy due to the overloading of the word type. Let's suppose for a moment that, in the future, Python supported some sort of "type" (ugh) annotations. Then you might have an interface like: interface Foo: def XInterface spam(YInterface a) Now here we have assigned a return "type" and an argument "type" but we haven't placed any restrictions on the implementation of the argument or return value. OTOH, even if we had: interface Foo2: def int spam(float a) where 'int' and 'float' named specific implementations, we *still* haven't said anything about the implementation of objects that implement Foo2. Of course, objects that implement Foo2 are less flexibly wrt to their arguments, but that may be acceptable because the lack of flexibility is compensated for by some higher level of service. > But... a 'non-specific' type is just a point with no attributes: > the same as the next type! So how is this a 'type system'? What do you mean by a type system? Some people refer to the current Python "type" and "class" implementations as a "type system". They are speaking mostly about implementation details. The Interface proposal seeks to introduce a behavioral type system so that objects can be composed without having to query specific implementation details. > That's the whole point. You cannot get 'polymorphism' UNLESS > you abstract away ALL the properties of objects. I disagree. For example: def render(device, ob): assert (implements(device, DeviceInterface) and implements(ob, DrawableInterface)) Here the render method imposes pretty specific restrictions on it's arguments. It expects devices and drawables to implement specific interfaces which are used in the render method. OTOH, the render method is highly polymorphic. Devices might be CRTs, Postscript files, pen plotters, whatever. Drawables might include geometric shapes, maps, etc. > You must vest > all specification in the relationships of methods with each other. You may find this approach useful, but I don't think it is the only approach. > I will to give a simple but "real world" example: a stack. > A stack has methods 'push' and 'pop'. It is characterised by the rule: > > push . pop = identity > > (roughly -- there is an _exact_ but more complex set of rules that > accounts for empty stacks and stack overflow which I will ignore for clarity). You're right, this *is* a simple example. Formal systems can be very entertaining for such simple examples, but it appears to me that they don't scale very will. What happens to the example above if the stack is bounded. Also, the statement above doesn't tell me, at least not directly what I consider to be some very useful facts, including: - What arguments does push take? I'm not taking "types" here. I'd just like to know whether I have to pass one argument, two, none, etc. - Why would I want to call push in the first place? - Does push return anything? - Same as above for pop. - What is the relationship between the argument(s) passed to push and the value returned from pop? - Why do I care about identity? Frankly, I'd find the following interface to be more useful: interface Stack: """First-in last-out object collections. """ \ def push(x): """Add the argument x to the stack The object may be retrieved later with the pop method. Objects are retrieved in first-in last-out order. No value is returned. """ def pop(): """Remove and return an object from the stack. The object in the stack that was most recently added is returned. """ > The exposition above is important because unlike, for example, > a C++ template class, it doesn't even name the type of the object > being pushed. This is a good point. I certainly consider C++ and Java's "type systems" to be kind of disgusting. ;) An interface system that doesn't make you say things that you don't want to say is a good thing, IMO. Note that in the informal interface definition I gave above, I didn't have to say anything about the objects added to the stack either. Unlike your example, I was able to say something that you failed to say, which was that aobjects are actually added to and removed from the stack. > In a C++ stack template, the type is named but arbitrary: > this is equivalent but more verbose. It is _always_ possible to > express it without mentioning the type, like above. > > Not mentioning the type, even as a 'variable' is not necessary > logically, but it is very necessary for a different reason: > it provides a _syntactic_ guarrantee of abstraction. > If the language constructions don't let you name the type, > you cannot get any 'implementation details' into the specification > for the simple reason there is no syntax for it! This is a good thing. OTOH, sometimes it is useful to be able to set behavioral or even implementation restrictions in the interface. > Just how abstract is the category above? > Isn't that what you want for an interface! No, it doesn't contain enough information. And it's not clear to me how to apply the same approach to make less trivial statements. > What is an instance? > > Answer: any other category X which is the image > of a functor from Stack. Such as that with the rule: > > push_int . pop_int = identity_int > > > A tempting addition would be to add behavioral information, like > > preconditions, postconditions, and invariants. The intension > > would be to have these conditions checked at run time. This is > > somewhat problematic, however, because many interesting conditions > > will depend on state. In a specification language, abstract state > > is used, but Python run-time checks would need to use concrete > > state. Using concrete state in an interface specification would > > violate the separation of interface and implementation. Then > > again, if completeness is not a goal, which I think it can't be, > > then many interesting conditions *could* be expressed without > > reference to concrete state. > > No, you are missing the point. I may be missing *your* point. I'm pretty sure there are some points I am able to grasp. Probably these include *my* points. > In Python, it is a matter > of _state_ whether an object is one type or another. There is no > hard distinction between the two. So checking type but not state > is relying on a distinction that doesn't exist. I strongly suspect that we (hm, well, maybe you) are running afoul of the badly overloaded term, "type". I didn't use the term type above. In my (limited) experience (basically, just some grad school courses), to completely model (capture behavior of) an abstract data type, you need to model abstract state. This would add alot of formality that would put it beyond the reach or interest of most programmers. Fortunately, formal completeness is a specific non-goal of my proposal. > You suggest that checking state violates the abstractness > of the specification and you're right. > > Checking TYPE is just as bad for the same reason. > Consider Euclids algorithm which computes the gcd of two > 'numbers'. Should we check the arguments are ints? I have tried, for good reason, to avoid the word "type" in my proposal. I suggest it would be a good idea to eliminate it from these discussions. I never said that signatures included anything smelling like interface or implementation information, although I do not think this would necessarily be a bad idea. I certainly don't mind using interface that impose behavioral restrictions on inputs. I don't even mind using interfaces that impose implementation restrictions on inputs if the lack of flexibility is compensated for by other features. Polymorphism is not my only goal. For example, I might not mind using an object that implemented an interface like: interface BigBadMath: def ArrayInterface invertHumoungousArray( DamnSpecificArrayOfFloatsClass x) ... if objects that implemented this interface could invert humoungous arrays 1000 times faster than objects that supported more flexible interfaces. (snip) > The fact that OO ties these methods to the arguments > is unequivocably wrong. a. OO does nothing of the sort. Certain OO and non OO languages make you say things you don't want to say about arguments and return values. Other OO (and non-OO) languages don't. b. It is useful at times to be able to restrict arguments, as I have tried to demonstrate in examples above. > Lets not compound the error by repeating > Meyer's mistake. > > ----------------------------------- > > Here's my approach. First, I have implemented a protocols module > in interscript. Get interscript, if only to examine that one module. > Or examine the documentation at: > > http://www.triode.net.au/~skaller/interscript > > and look for the protocols module in the core subpackage of the > implementation section of the document. I did. > This module just allows objects to be 'tagged' with the name of > a protocol such as 'sequence' 'file' or 'mapping'. > It has some conveniences such as class protocols, reflection > of the protocols of bases, and using the names of types as > protocols. But in essence the module does nothing more than > allow tagging objects with protocols. > > This _clearly_ is not enough because it doesn't provide any > mechanism to express relationships between protocols. > (Reflection of inheritance is a convenient hack!) It is not only a convenient hack, it reflects the way many people think about the world (classification) and it is a model that is already familiar to Python programmers. > But it has an advantage -- it is implemented right now, > it requires no language extensions, and it covers a lot of > basic territory including the whole of the existing type > system and all the defined and more abstract protocols. As stated in the proposal, the Scarecrow proposal also doesn't require any language changes, and FWIW the implementation is trivial. It also provides: - Organization of interfaces in an interface "hierarchy", which is, of course, really a DAG, - Interface objects, rather than just string tags, that provide a basis for managing and using meta data, like doc strings, (at least minimal "generic") signatures, pre-and post conditions, whatever.... > Now, according to category theory, the _meaning_ of a protocol > cannot be embodied in the protocol -- it is just an arbitrary label. > It is the relationships between protocols that are important. Ok, you've talked me into not using category theory. Fortunately, I didn't use it in the Scarecrow proposal. ;) > So what is required -- given the protocol tagging mechanism I've > implemented -- is a way of describing these relationships. > > The 'is-a' relationship is not be ignored, even though it isn't everything. > If one can say Protocol A implies Protocol B, this is important information > (even if it is not fundamental). > > Example: mutable sequence implies sequence. > > Another construction is composition: if one can say a collection of functions > obeys protocol A and also protocol B, then it obeys protocol C, this is also > important information. For example: > > Mutable and sequence implies mutable sequence. > > It's fairly obvious that by fixing the object (set of methods!!) we're just dealing > with predicates and restricted first order logic. (No negation!) Deductions in this > system can be trivially handled by existing python. [More complex deductions > can be done by logic systems like Prolog, or better, Mercury] > > What is much more interesting is _intra-object_protocol: protocols obeyed by > two, three, or familiies of objects NOTof the same 'type'. I assume that by 'type' you mean type in the formal sense (ADT). Yes, that would be interesting. This can be captured, at least partially through restrictions in the interfaces themselves. For example: interface Renderer: def render(display, drawable): assert (implements(display, Display and implements(drawable, Drawable) does express an interrelation between the Rendered, Display, and Drawable interfaces. > These protocols _cannot_ be expressed by tagging objects because the set of > methods obeying the protocol aren't all methods of the same class. I think I did express a mult-interface protocol above through an essentially tagging mechanism. > For example: a list constructed with a list header and nodes has two types, > and the protocols involved in managing the collection are functions > which may operate on several nodes as well as the header. > > We can specify the protocol by writing global functions > like insert and delete, and defining a _category_: that is, specifying > the relationships between these methods directly. > > In this scenario, if one writes a generic algorirthm, one passes _functions_ > to it as arguments, and one must test that the function stand in the relationship > defined by a category if one is to determine whether the algorithm will work. > > Example: > > def dup(push, pop): > x = pop() > push(x) > push(x) > > will duplicate the top element of a stack. What is the requirement? > Obviously, 'it' has to 'be' a stack! What does that mean?? > Do you remember?? > > push.pop = identity > > So what is 'the stack'?? It is the pair of functions (push, pop). It isn't an object!! > > I am developing the technology to allow construction of categories in Python. > Here's roughly what I see happening: > > 1) construct an abstract stack category 'stack'. The members, push, pop, and identity > are class instances with no attributes. The category records the rule push.pop = identity > in a table. > > 2) Attach to each arrow of the category an attribute named 'F' whose value is > a function. F is a functor. The assertion which must be tested is that for all x: > > s = stack.push.F (s,x) > stack.pop.F(s) == x > > ['stack' is not the stack, s is the stack. 'stack' is the abstract category!] > > A routine is passed the functor F by passing: > > stack, 'F' > > ie, the abstract category together with the string name of the functor. > The routine doesn't check anything. It can't: the universal quantifier > is involved here.(for all x). But there is an implicit assertion that F > is a functor and it COMPLETELY DEFINES ALL CONSTRAINTS ON > THE INSTANCES stack.pop.F and stack.push.F. > > This is the crucial point: the specification is complete, formal, and also > available at run time (even though it cannot be proven). > Note that it can be _disproven_ by a single case, and so is > amenable to testing. > > So it isn't possible to check everything but it _is_ possible to > provide an utterly formal complete specification. > > In all realistic cases, proof of correctness is possible, > in some cases it may be possible to verify the proof, > and in some it may be possible for an inference engine > to deduce it with limited or no assistance. I think this > is the 'best case' scenario: complete formal interface > specifications with the potential for mechanical verification > in some cases, possibly with assistance, by existing > logic engines. People have built languages for doing this sort of thing. Why don't you use one of these languages? You don't seem to like OO. OK. I do find OO to be very useful. I also think formal languages are kind of fun too. I've had some fun with Z. Formal languages can be useful in "critical systems" where you just absolutely have to get algorithms right. Formal systems are really pretty darn hard for not trivial cases and require a good bit of training. I don't think that these are tools that are suitable for the vast majority of Python programmers. Also note that you don't have to turn Python into a formal specification language to use a formal specification language to model algorithms that get implemented in Python. You can use various formal languages to reason about and even provide the correctness of algorithms and then map those algorithms into Python. > The _difficult_ part of all this is integrating the most trivial > cases -- tagging objects and classes -- with the more > general cases, and it is hampered by the lack of > operators that act on function objects in python. > Category theory tells what is required. Only 10% of it is there. > For example, there is no way to simple compose two functions. > It can be emulated with a class .. but it should be a fundamental > primative. Have fun. I'm interested to see what you come up with. I'd be a lot more interested if you applied this energy to more OO formal systems, as I think that there'd be a higher payoff for Python. Jim -- Jim Fulton mailto:jim@digicool.com Technical Director (540) 371-6909 Python Powered! Digital Creations http://www.digicool.com http://www.python.org Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats. From M.Faassen@vet.uu.nl Tue Nov 24 15:22:33 1998 From: M.Faassen@vet.uu.nl (Martijn Faassen) Date: Tue, 24 Nov 1998 16:22:33 +0100 Subject: [Types-sig] Scarecrow Proposal References: <3657ECA1.6B2BD52@lemburg.com> <365A0634.9D7B079F@digicool.com> <365A7E58.7111F703@lemburg.com> <365AAE8D.6D38911B@digicool.com> Message-ID: <365ACF39.B38596BF@pop.vet.uu.nl> Jim Fulton wrote: > > M.-A. Lemburg wrote: > > > > > > To be honest: I just flew over it at the time and then decided it > > was a good thing but not something that would interest me too much. > > After a while I started to see that this could be the solution to > > providing "static" typing in a very smart way (with interfaces being > > used as "type" tags providing all kinds of nifty details to possible > > optimizing engines > > Perhaps someone will prove me wrong, but I expect that optimization > engines would want certain implementation information that would > be absent from a purely behavioral specification. > > Hm. Maybe it would be worth consider additional meta objects > (other than interfaces) that could be attributed to classes > or objects to give optimization hints. Let me try to figure out some of the conceptuology involved in all the various Types-SIG issues and list them, hope it helps: --- optimization hint or "hint" --- * Used by a Python interpreter or compiler to optimize Python code. * Can conform to an interface (?) (but an interface can't *be* a hint) --- interface (or protocol, or behavior) --- * Used to automatically ensure objects conform to certain behaviors, before starting to use them in this particular role (role! another name for interface!). * Can inherit interface specifications from other interfaces. Automatic parts of an interface could be: - method signatures (can check automatically if these exist in a class or object) - pre/post conditions (can check automatically if a particular object conforms to these) Other parts of an interface: - documentation (many things can't be automated but it would be nice if they were spelled out) --- type --- * Used to automatically ensure variables are of a certain behavior, before starting to use them in that role. This is useful for two reasons: * Ensure program correctness (users of code should pass arguments of the right type to be able to work with them) * Ensure the program works at all (a string type in the place of a number type usually makes no sense and would cause a crash if this weren't checked in a C extension) * Can conform to an interface In the case of *dynamic* types, we check them at run-time. The optional *static* types would enable compile time checks to take place in some circumstances. Static types are also easier to use for optimization purposes than dynamic ones. So, another reason for types (especially static types) might be: * Can be used to optimize Python code --- class --- * Used to automatically ensure variables are of a certain behavior, before starting to use them in that role. Useful because: * Ensure program correctness * Can inherit actual implementation from other classes * Can conform to an interface Classes currently are not types, but one future goal is to get rid of this split. --------------- How do all these interrelate? Is a hint a type? A static type? Is an interface a type? In Scarecrow it's something like a class (I'm not sure I get the exact relation between class an interface in the Scarecrow proposal), but since classes will eventually hopefully be types, we'll have to assume an interface would become a type. Classes and interfaces cannot generally be used for optimization purposes; an interface is a pure behaviorial specification and we *want* to be able to have an integer interface that is not an actual hardware integer, but can front for classes too. Types in general, static or not, cannot be used directly for optimization purposes either, as these will include interfaces and classes. This is why 'hints' are a useful concept to have as well in this discussion. Is the following 'inheritance tree' true in Python 2? type (static or dynamic) / | \ (isa) interface class hint We should be careful we don't complicate things; we need to figure out what actually can be optimization hints, and what is used to ensure program correctness, and how they all fit together. I know that is what this SIG is for, so sorry for restating it. I also know the current goal is Scarecrow, and what I discuss lies further in the future. We should keep all these issues clear however. If we don't, when everyone adopts Scarecrow 2.0, we might run into awful problems trying to do Python 2.0 optimization and optional static type checking. We wouldn't want people to have to hack with, through, and around the interface system to do static typing or optimization. We want everything to be smooth, clear and elegant. :) I hope I didn't get everything confused, and that I'm somewhat comprehensible. Most of all I hope I managed to contribute something to this discussion. Martijn From Justus Pendleton Tue Nov 24 16:24:39 1998 From: Justus Pendleton (Justus Pendleton) Date: Tue, 24 Nov 1998 11:24:39 -0500 Subject: [Types-sig] Re: PRE-PROPOSAL: Python Interfaces In-Reply-To: <365AD21E.F25A4EAA@digicool.com>; from Jim Fulton on Tue, Nov 24, 1998 at 03:34:54PM +0000 References: <3642E80C.4A997D94@digicool.com> <364b1070.432371637@news.triode.net.au> <365AD21E.F25A4EAA@digicool.com> Message-ID: <19981124112439.A1091@ryoohki> On Tue, Nov 24, 1998 at 03:34:54PM +0000, Jim Fulton wrote: > Also note that you don't have to turn Python into a > formal specification language to use a formal specification > language to model algorithms that get implemented in Python. > You can use various formal languages to reason about and > even provide the correctness of algorithms and then map > those algorithms into Python. While I think the discussion about types (and interfaces and whatnot) is (occassionally) illuminating and (sometimes) interesting, I can't help but wonder if all the effort is misplaced. Python seems to have done just fine without interfaces. Lots of languages have done fine without interfaces. While I happen to think that interfaces are nifty I can't recall ever seeing any empirical evidence that they are useful in any way whatsoever. What's more, there are plenty of languages that already do most (if not all) of the things that people are asking to have Python do. Why not just use one of those languages? While I think that Python is a nifty language, it isn't exactly popular. What popularity it _does_ have is, I think, because of its rapid prototyping abilities. I wonder if all of these proposals are trying to turn a loose language into some hulking Ada95 clone.... -- Justus Pendleton From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Tue Nov 24 17:19:29 1998 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Tue, 24 Nov 1998 12:19:29 -0500 (EST) Subject: [Types-sig] Re: PRE-PROPOSAL: Python Interfaces References: <365700BD.FE8B1776@technologist.com> <1300449328-21246608@hypernet.com> Message-ID: <13914.60065.348794.164204@anthem.cnri.reston.va.us> [I'm chopping the recip list to just types-sig -BAW] >>>>> "GM" == Gordon McMillan writes: GM> In fact (as Barry was supposed to point out at the conference GM> for me, harrumph, harrumph), the set of all interfaces / GM> protocols is the set of all subsets of the set of all GM> signatures - without taking behavior into account. That is, GM> the set of all interfaces has a cardinality that I don't GM> particularly want to contemplate. Yes, my fault for not bringing that up, but you've done a fine job in my lapse! >>>>> "TP" == Tim Peters writes: TP> OTOH, unlike e.g. JimF I have nothing against stuffing some TP> default implementation into interfaces. To the contrary, if I TP> *wanted* to say that all List implementers implement Stack TP> too, I think In fact, I want a different kind of error to get thrown when a class says it implements a particular interface, but in fact neglects to. If I have a class List that claims to implement interface Stack, but in fact does not provide an implementation of pop() or push(), I don't want an AttributeError when I call one of those methods on an instance of List. I want something like a NotImplementedError. I think this important enough to guarantee in language spec. Tim also wants to allow implementation in an interface definition, because some methods can be defined solely in terms of other methods in the interface. That seems fine to me. GM> True. One would not even try to enumerate all possible GM> signatures. The point is, though, that there's a conflict GM> between precision and usability. If interfaces are too GM> precise, they will multiply with abandon and become GM> unmanageable. To prevent that, people will choose the GM> "closest" interface. So the "requirer" will ask for more than GM> is actually needed, and the "requiree" will have the choice of GM> adding unneeded methods, wrapping unnecessarily, or lying. This is a good point, and I'm reminded of a discussion that Guido and I had when we were talking about the class-exceptions heirarchy. We'd drawn up several possibilities, some quite complicated, but we had too much trouble assigning the proper semantics to the classes that we decided to flatten the heirarchy as much as possible, and leave it to applications to decide on deeper heirarchies. I think the same approach will have to be taken with the core w.r.t. interfaces. Keep the set of interfaces defined by the core as shallow and small as possible, but provide enough rich mechanisms so that applications can build higher level stuff easily. -Barry From mal@lemburg.com Tue Nov 24 18:58:47 1998 From: mal@lemburg.com (M.-A. Lemburg) Date: Tue, 24 Nov 1998 19:58:47 +0100 Subject: [Types-sig] Scarecrow Proposal References: <3657ECA1.6B2BD52@lemburg.com> <365A0634.9D7B079F@digicool.com> <365A7E58.7111F703@lemburg.com> <365AAE8D.6D38911B@digicool.com> Message-ID: <365B01E7.55628B98@lemburg.com> Jim Fulton wrote: > > M.-A. Lemburg wrote: > > > > > > To be honest: I just flew over it at the time and then decided it > > was a good thing but not something that would interest me too much. > > After a while I started to see that this could be the solution to > > providing "static" typing in a very smart way (with interfaces being > > used as "type" tags providing all kinds of nifty details to possible > > optimizing engines > > Perhaps someone will prove me wrong, but I expect that optimization > engines would want certain implementation information that would > be absent from a purely behavioral specification. > > Hm. Maybe it would be worth consider additional meta objects > (other than interfaces) that could be attributed to classes > or objects to give optimization hints. Actually, I thought of doing all those things (behaviour, optimization hints and pre-/post-conditions) with interfaces. After all, it's intended to provide structured meta-data about objects and all of the three types of information fit in there nicely. > > IMHO, interfaces ought to have information query methods, not > > "real" ones that are only given to describe which methods are > > expected to be implemented by a class conforming to an interface. > > I agree. > > > I would put more emphasis on the tagging nature of interfaces > > rather than making them look like a C++ base class. > > I couldn't agree more. > > > [Code sample] > > I'll post me initial implementation later today. > This should make things a little bit clearer. Looking forward to it. These discussions are getting a little hard to handle without proof-of-concept implementations (I believe that Python already has all necessary features to do so). > > But I could define an Integer interface to mean: objects conforming > > to this interface will work fine with the builtin int() and the > > optimizer may convert them using int() to enable better ways of > > storing their values. (The object could also define a reverse > > method, e.g. reinit_from_int() to reverse that trick -- all well > > definable in an interface definition.) > > Fair enough. The interface should probably say, if it can, what the > possible range of values is. (e.g. can a 16-bit integer be used at the > C level, must a python long be used, etc.) Right. -- Marc-Andre Lemburg Y2000: 402 days left --------------------------------------------------------------------- : Python Pages >>> http://starship.skyport.net/~lemburg/ : --------------------------------------------------------- From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Tue Nov 24 22:55:50 1998 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Tue, 24 Nov 1998 17:55:50 -0500 (EST) Subject: [Types-sig] Scarecrow Proposal References: <3657ECA1.6B2BD52@lemburg.com> <365A0634.9D7B079F@digicool.com> <365A7E58.7111F703@lemburg.com> <365AAE8D.6D38911B@digicool.com> Message-ID: <13915.14710.82888.768798@anthem.cnri.reston.va.us> JF> Note that signatures will *not* be available as attributes of JF> the interface. For example: | class FooInterface(Interface): | def spam(a, b): | "blah blah blah :)" JF> FooInterface.spam # raises an attribute error Why? Doesn't it make sense for introspection purposes to have access to the methods defined in an interface through the interface object? -Barry From tim_one@email.msn.com Tue Nov 24 23:18:38 1998 From: tim_one@email.msn.com (Tim Peters) Date: Tue, 24 Nov 1998 18:18:38 -0500 Subject: [Types-sig] Re: PRE-PROPOSAL: Python Interfaces In-Reply-To: <13914.60065.348794.164204@anthem.cnri.reston.va.us> Message-ID: <000901be1800$c48a09c0$ab9e2299@tim> [Barry A. Warsaw] > ... > I want a different kind of error to get thrown when a class > says it implements a particular interface, but in fact neglects > to. If I have a class List that claims to implement interface > Stack, but in fact does not provide an implementation of pop() > or push(), I don't want an AttributeError when I call one of > those methods on an instance of List. I want something like > NotImplementedError. I think this important enough to guarantee > in language spec. > > Tim also wants to allow implementation in an interface > definition, because some methods can be defined solely in terms > of other methods in the interface. That seems fine to me. As we discussed on the egroups version of this, if interfaces allow supplying impementation, then from InterfaceSupport import deferred class Stack(Interface): def push(self, thing): deferred() etc would make everyone who *claims* to implement Stack pick up the (presumably error-raising) Stack.push "deferred()" implementation if they don't supply their own (or inherit some other) real implementation of push. > ... > I think the same approach will have to be taken with the core > w.r.t. interfaces. Keep the set of interfaces defined by the > core as shallow and small as possible, but provide enough rich > mechanisms so that applications can build higher level stuff > easily. I expect the core will end up with a shallow but large set of interface tags, because tiny pieces of the implementation get reused in funky ways in the current folklore interfaces. Like "for x in thing" requires a thing.__getitem__ that accepts an integer argument and either returns a value or raises IndexError, but uses no other properties of "a sequence" (not even __len__!). That little piece should be given a name, and the Sequence interface should specifically say that it implements it. Similarly. whether a thing supports "len(thing)" is a teensy piece of behavior that's not unique to Sequence (e.g. Map supports it too, and any number of user-defined classes that aren't intended to be Sequence or Map). Maybe we should name these mini-interfaces HasAttr__getitem__ and HasAttr__len__ . healing-the-type/class-split-would-make-it-simpler-ly y'rs - tim From tim_one@email.msn.com Tue Nov 24 23:18:34 1998 From: tim_one@email.msn.com (Tim Peters) Date: Tue, 24 Nov 1998 18:18:34 -0500 Subject: [Types-sig] "Interfaces" Jargon In-Reply-To: <3659AA1F.D220006E@digicool.com> Message-ID: <000601be1800$c1b0ab00$ab9e2299@tim> [Jim Fulton] > ... > At the "Python Interfaces" session at Developer's Day '98, I > took a straw pole and most respondents seemed to favor the > terms "interfaces" and "implements", including Guido, Jim H., > and myself, FWIW. And here I was all set to play along, until you revealed that The Establishment favors these names! I'll just pretend that Guido, JimH and yourself hate those names, so I can feel like a rebel again. words-make-a-fickle-master-ly y'rs - tim From jim@Digicool.com Tue Nov 24 23:39:30 1998 From: jim@Digicool.com (Jim Fulton) Date: Tue, 24 Nov 1998 23:39:30 +0000 Subject: [Types-sig] Scarecrow Proposal References: <3657ECA1.6B2BD52@lemburg.com> <365A0634.9D7B079F@digicool.com> <365A7E58.7111F703@lemburg.com> <365AAE8D.6D38911B@digicool.com> <13915.14710.82888.768798@anthem.cnri.reston.va.us> Message-ID: <365B43B2.EAFE1331@digicool.com> Barry A. Warsaw wrote: > > JF> Note that signatures will *not* be available as attributes of > JF> the interface. For example: > > | class FooInterface(Interface): > | def spam(a, b): > | "blah blah blah :)" > > JF> FooInterface.spam # raises an attribute error > > Why? Doesn't it make sense for introspection purposes to have access > to the methods defined in an interface through the interface object? Of course, but *not* as attributes of the interface. The interface will provide methods for getting these. Jim -- Jim Fulton mailto:jim@digicool.com Technical Director (540) 371-6909 Python Powered! Digital Creations http://www.digicool.com http://www.python.org Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats. From just@letterror.com Tue Nov 24 23:52:32 1998 From: just@letterror.com (Just van Rossum) Date: Wed, 25 Nov 1998 00:52:32 +0100 Subject: [Types-sig] Scarecrow Proposal In-Reply-To: <13915.14710.82888.768798@anthem.cnri.reston.va.us> References: <3657ECA1.6B2BD52@lemburg.com> <365A0634.9D7B079F@digicool.com> <365A7E58.7111F703@lemburg.com> <365AAE8D.6D38911B@digicool.com> Message-ID: At 5:55 PM -0500 11/24/98, Barry A. Warsaw wrote: > JF> Note that signatures will *not* be available as attributes of > JF> the interface. For example: > > | class FooInterface(Interface): > | def spam(a, b): > | "blah blah blah :)" > > JF> FooInterface.spam # raises an attribute error > >Why? Doesn't it make sense for introspection purposes to have access >to the methods defined in an interface through the interface object? Absolutely. I was even thinking if inheriting __doc__ string could be a feature: interface FooInterface: def spam(a, b): "blah blah blah." class FooInterface FooClass: def spam(self, a, b): pass >>> f = FooClass() >>> f.spam.__doc__ 'blah blah blah.' >>> Maybe too far-sought... Just From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Wed Nov 25 00:24:41 1998 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Tue, 24 Nov 1998 19:24:41 -0500 (EST) Subject: [Types-sig] Re: PRE-PROPOSAL: Python Interfaces References: <13914.60065.348794.164204@anthem.cnri.reston.va.us> <000901be1800$c48a09c0$ab9e2299@tim> Message-ID: <13915.20041.23787.219850@anthem.cnri.reston.va.us> >>>>> "TP" == Tim Peters writes: TP> As we discussed on the egroups version of this, if interfaces TP> allow supplying impementation, then | from InterfaceSupport import deferred | class Stack(Interface): | def push(self, thing): deferred() | etc TP> would make everyone who *claims* to implement Stack pick up TP> the (presumably error-raising) Stack.push "deferred()" TP> implementation if they don't supply their own (or inherit some TP> other) real implementation of push. That definitely does the trick, assuming we add something like a NotImplementedError to the core and perhaps even a builtin function deferred() that just raises the exception. Half of me wants something a little more magical (and thus less dependent on the author to play the right game)... okay make that 1/4 of me.. no 1/8... urg. -Barry From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Wed Nov 25 00:28:17 1998 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Tue, 24 Nov 1998 19:28:17 -0500 (EST) Subject: [Types-sig] Scarecrow Proposal References: <3657ECA1.6B2BD52@lemburg.com> <365A0634.9D7B079F@digicool.com> <365A7E58.7111F703@lemburg.com> <365AAE8D.6D38911B@digicool.com> <13915.14710.82888.768798@anthem.cnri.reston.va.us> <365B43B2.EAFE1331@digicool.com> Message-ID: <13915.20257.865977.315618@anthem.cnri.reston.va.us> >>>>> "JF" == Jim Fulton writes: >> Barry A. Warsaw wrote: >> Doesn't it make sense for introspection purposes to have access >> to the methods defined in an interface through the interface >> object? JF> Of course, but *not* as attributes of the interface. The JF> interface will provide methods for getting these. Okay, but then the objects those methods return will have to have all the (appropriate) attributes one might expect to be able to get from a "normal" method. -Barry From janssen@parc.xerox.com Wed Nov 25 01:35:53 1998 From: janssen@parc.xerox.com (Bill Janssen) Date: Tue, 24 Nov 1998 17:35:53 PST Subject: [Types-sig] Optional static typing and Python interfaces In-Reply-To: <3659EC1A.8EDD4676@digicool.com> References: <3656BB11.641FE3ED@technologist.com> <1300476788-19595006@hypernet.com> <3659EC1A.8EDD4676@digicool.com> Message-ID: Excerpts from ext.python: 23-Nov-98 Re: PRE-PROPOSAL: Python In.. Jim Fulton@digicool.com (1966*) > I think they are two different problems and I expect different solutions. > They need not be competing. One might be useful to the other. For example, in the proposal we were bouncing around 3 years ago, the progression of adding optional static typing went: 1) Add interface descriptions (.pi files) which used a slightly different syntax in that they allowed some kind of optional static typing declarations. These serve as documentation; no compilers need muck with them. Work out the details of the optional static typing syntax at this stage. 2) Add a program which takes module descriptions for some modules, along with some Python code which uses those modules, and does some type inferencing on the code to find detectable (not all) errors in uses of the module. It can also check for other errors, such as missing attributes. 3) Modify the Python parser to allow optional static typing in the Python code, using the syntax developed in phase 1. Beef up the cross-checking program developed in phase 2 to use these type declarations to aid it in type inferencing. Bill From tim_one@email.msn.com Wed Nov 25 02:06:43 1998 From: tim_one@email.msn.com (Tim Peters) Date: Tue, 24 Nov 1998 21:06:43 -0500 Subject: [Types-sig] Scarecrow Proposal In-Reply-To: <13915.14710.82888.768798@anthem.cnri.reston.va.us> Message-ID: <000201be1818$3f219600$9e9e2299@tim> [JimF] > Note that signatures will *not* be available as attributes of > the interface. For example: > > class FooInterface(Interface): > def spam(a, b): > "blah blah blah :)" > > FooInterface.spam # raises an attribute error [BarryW] > Why? Doesn't it make sense for introspection purposes to > have access to the methods defined in an interface through > the interface object? Indeed, after 5 years of enhancing these things, if we're lucky we may make them almost as powerful as classes again . misunderstandingly y'rs - tim From tim_one@email.msn.com Wed Nov 25 06:12:47 1998 From: tim_one@email.msn.com (Tim Peters) Date: Wed, 25 Nov 1998 01:12:47 -0500 Subject: [Types-sig] RE: PRE-PROPOSAL: Python Interfaces In-Reply-To: <1300347251-642173@hypernet.com> Message-ID: <000401be183a$9fa1bf60$6d9e2299@tim> [Uncle Timmy] > ... > IOW, Lists do *not* implement the Stack protocol just > because "it looks like" they do -- if they do at all, > it's only by virtue of List explicitly saying it implements > specifically Stack. Today, though, there's no way to draw > the distinction at all. [Gordo the Magnificent] > Which brings up the point Paul made that if the all-omiscient > Guido forgot to say that (the builtin) List implements Stack, > and his time machine will have been in the body shop because > of that fender-bender, then I'd better be able to say at > runtime that "I know this is a List, but you'll treat it like > a Stack anyway". This all made more sense in the egroups context, where part of healing the type/class split was widely taken to be erasing "needless" distinctions between builtin and user-defined clypes (as good a name as any for a hybrid class/type -- well, OK, really much worse than any other name I can think of! so I'll stick with it). In *that* world, you have three more choices: 1) If Guido only forgot to add Stack's push method to List, you can add your own via subclassing: clype Slack(List): __implements__ = Stack def push(self, thing): self.append(thing) and pass a Slack to the Stack consumer. 2) Or add it to List directly if the Stack consumer isn't relying on interfaces but on ad hoc hasattr tricks: List.push = lambda self, append: self.append(thing) 2) If Guido did implement all the Stack methods in List, and simply forgot to anticipate the Stack interface, you could repair his failed precognition yourself via e.g. List.__implements__.append(Stack) No time machine required -- except maybe one that moves forward beyond today's clypesplit. >> OTOH, unlike e.g. JimF I have nothing against stuffing some >> default implementation into interfaces. > Yeah. Somehow my C++ "abstract base classes" always end up not > being abstract. I have more the Eiffel class model in mind, which makes no distinction at all between classes and interfaces, behavior and specification, abstract and concrete -- except in that the built-in doc tools generate different *views* of a class depending on what you tell it to show you. If you want an Eiffel class that describes only interface, fine -- just "defer" all the methods. If some methods can be implemented in terms of others, fine too -- supply implementations for those methods and defer the rest. The one thing Eiffel can do here Python can't (short of the more esoteric stuff like automagically adjusted method pre/post-conditions and class invariants) is say "ya, I know I inherited Stack, but I don't want *my* kids to inherit Stack too unless they do it themselves". Which addresses JimF's biggest gripe with class inheritance. I see the Scarecrow Proposal as fixing that for Interfaces but leaving the problem untouched for Classes; Eiffel solves both at the same time via a single mechanism. >> [point 14 from Timmy's egroups manifesto] > Which Basilica door did you nail this to? (IE, where can I > read it?) This stuff should really be moved to the Types SIG. It's at http://www.egroups.com/list/python-classes/ although you probably need to go to the top page first and register for a (free) account. When you finally get to Python Classes, click the "by Date" button (the thread view is useless), and start at the oldest. Things get interesting when the various "position papers" show up, and then get confused. According to my Inbox, there are 40 msgs from that group I still haven't made time to read. But they probably solve everything . > ... > I will admit to being completely mystified by the following > (from lib/module-select.html): > > "You may also define a wrapper class yourself, as long as it > has an appropriate fileno() method (that really returns a > Unix file descriptor, not just a random integer)." > > particularly-on-WIndows-ly y'rs See? Two months for now it can simply say: You may also define a wrapper class yourself, implementing the UNIXFileNo interface. In this way we turn all confusions into Fred Drake's problem, who merely needs to supply an insightful writeup for UNIXFileNo's index entry to point at <0.7 wink>. but-you'll-have-a-name-for-the-missing-info- as-well-as-a-natural-place-to-put-it-ly y'rs - tim From fredrik@pythonware.com Wed Nov 25 10:08:20 1998 From: fredrik@pythonware.com (Fredrik Lundh) Date: Wed, 25 Nov 1998 11:08:20 +0100 Subject: [Types-sig] "Interfaces" Jargon Message-ID: <016601be1861$39421900$f29b12c2@pythonware.com> >[Jim Fulton] >> ... >> At the "Python Interfaces" session at Developer's Day '98, I >> took a straw pole and most respondents seemed to favor the >> terms "interfaces" and "implements", including Guido, Jim H., >> and myself, FWIW. Hmm. I seem to recall that the result was 19 votes for interfaces and 17 votes for behaviour, or something not very far that. And the term "behaviour" was introduced just a few minutes before the poll... Cheers /F From fredrik@pythonware.com Wed Nov 25 10:08:20 1998 From: fredrik@pythonware.com (Fredrik Lundh) Date: Wed, 25 Nov 1998 11:08:20 +0100 Subject: [Types-sig] "Interfaces" Jargon Message-ID: <016901be1861$40af09a0$f29b12c2@pythonware.com> >[Jim Fulton] >> ... >> At the "Python Interfaces" session at Developer's Day '98, I >> took a straw pole and most respondents seemed to favor the >> terms "interfaces" and "implements", including Guido, Jim H., >> and myself, FWIW. Hmm. I seem to recall that the result was 19 votes for interfaces and 17 votes for behaviour, or something not very far that. And the term "behaviour" was introduced just a few minutes before the poll... Cheers /F From jim@Digicool.com Wed Nov 25 11:48:36 1998 From: jim@Digicool.com (Jim Fulton) Date: Wed, 25 Nov 1998 11:48:36 +0000 Subject: [Types-sig] Scarecrow Proposal References: <3657ECA1.6B2BD52@lemburg.com> <365A0634.9D7B079F@digicool.com> <365A7E58.7111F703@lemburg.com> <365AAE8D.6D38911B@digicool.com> Message-ID: <365BEE94.9675822@digicool.com> Jim Fulton wrote: > > I'll post me initial implementation later today. > This should make things a little bit clearer. Well, that didn't happen. The actual machinery, using the meta-class hook was pretty simple, but I decided I wanted to fix a wart here and a wart there and .... Also, I'm going to be away for the next few days, so I'm going to wait till Monday to release my first cut. Sorry about that. Jim -- Jim Fulton mailto:jim@digicool.com Technical Director (540) 371-6909 Python Powered! Digital Creations http://www.digicool.com http://www.python.org Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats. From Justus Pendleton Wed Nov 25 14:51:08 1998 From: Justus Pendleton (Justus Pendleton) Date: Wed, 25 Nov 1998 09:51:08 -0500 Subject: [Types-sig] "Interfaces" Jargon In-Reply-To: <016601be1861$39421900$f29b12c2@pythonware.com>; from Fredrik Lundh on Wed, Nov 25, 1998 at 11:08:20AM +0100 References: <016601be1861$39421900$f29b12c2@pythonware.com> Message-ID: <19981125095108.B371@ryoohki> On Wed, Nov 25, 1998 at 11:08:20AM +0100, Fredrik Lundh wrote: > Hmm. I seem to recall that the result was 19 votes for > interfaces and 17 votes for behaviour, or something not > very far that. And the term "behaviour" was introduced > just a few minutes before the poll... You couldn't possibly use "behaviour" ... it would _have_ to be "behavior" :) -- Justus Pendleton From mal@lemburg.com Wed Nov 25 16:26:31 1998 From: mal@lemburg.com (M.-A. Lemburg) Date: Wed, 25 Nov 1998 17:26:31 +0100 Subject: [Types-sig] "Interfaces" Jargon References: <016601be1861$39421900$f29b12c2@pythonware.com> <19981125095108.B371@ryoohki> Message-ID: <365C2FB7.5CFE9A07@lemburg.com> Justus Pendleton wrote: > > On Wed, Nov 25, 1998 at 11:08:20AM +0100, Fredrik Lundh wrote: > > Hmm. I seem to recall that the result was 19 votes for > > interfaces and 17 votes for behaviour, or something not > > very far that. And the term "behaviour" was introduced > > just a few minutes before the poll... > > You couldn't possibly use "behaviour" ... it would _have_ to be "behavior" :) Another reason not to use that term :-) Since classes already have "class", why not add some "style" ? Types would look pretty dull in that glamorous company, though. -- Marc-Andre Lemburg Y2000: 401 days left --------------------------------------------------------------------- : Python Pages >>> http://starship.skyport.net/~lemburg/ : --------------------------------------------------------- From fredrik@pythonware.com Wed Nov 25 16:42:28 1998 From: fredrik@pythonware.com (Fredrik Lundh) Date: Wed, 25 Nov 1998 17:42:28 +0100 Subject: [Types-sig] "Interfaces" Jargon Message-ID: <00fd01be1892$97e79120$f29b12c2@pythonware.com> M.-A. Lemburg wrote: >Justus Pendleton wrote: >> On Wed, Nov 25, 1998 at 11:08:20AM +0100, Fredrik Lundh wrote: >> > Hmm. I seem to recall that the result was 19 votes for >> > interfaces and 17 votes for behaviour, or something not >> > very far that. And the term "behaviour" was introduced >> > just a few minutes before the poll... >> >> You couldn't possibly use "behaviour" ... it would _have_ to be "behavior" :) > >Another reason not to use that term :-) Hey, if Microsoft can translate the BASIC keywords to Swedish, Swahili, and whatever, we could at least allow for "european english" spellings in Python. Isn't that what the locale module is all about, btw? Cheers /F (no, I'm not serios) From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Wed Nov 25 18:50:25 1998 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Wed, 25 Nov 1998 13:50:25 -0500 (EST) Subject: [Types-sig] "Interfaces" Jargon References: <3659AA1F.D220006E@digicool.com> Message-ID: <13916.20849.249955.439005@anthem.cnri.reston.va.us> >>>>> "JF" == Jim Fulton writes: JF> I'd like to get moving on resolving jargon issues wrt to what JF> I like to call "interfaces". JF> If anyone *really* has a hard time using "interfaces" and JF> "implements" or if you think a complelling case can be made JF> against these or for others, then please speak up now. I personally don't like the term "interface" to describe these JimFultonThingies :-). I suppose my main arguments are that the class definition, which brings together all the specifications about how a concrete object behaves, via it's base classes and its conforms_to heirarchy, is also an interface. I also think the term "interface" is way too overloaded and that using that term means that people (like myself) bring whatever background baggage they have to that term. I prefer "protocol" and "conforms_to" but mostly from my ObjC background. "behavior" works for me, as does "discipline" although I think we've typically use the latter for the informal description of method signatures. But in reality the JavaJuggernaut has probably steamrolled this issue into decision already. I suspect since JimF will be the first one to the table with Working Code (tm), so his are the preferences that will eventually get adopted. So let's just decide and move on. -Barry From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Wed Nov 25 18:54:03 1998 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Wed, 25 Nov 1998 13:54:03 -0500 (EST) Subject: [Types-sig] Optional static typing and Python interfaces References: <3656BB11.641FE3ED@technologist.com> <1300476788-19595006@hypernet.com> <3659EC1A.8EDD4676@digicool.com> Message-ID: <13916.21067.434783.321083@anthem.cnri.reston.va.us> >>>>> "BJ" == Bill Janssen writes: BJ> 1) Add interface descriptions (.pi files) which used a BJ> slightly different syntax in that they allowed some kind of BJ> optional static typing declarations. These serve as BJ> documentation; no compilers need muck with them. Work out the BJ> details of the optional static typing syntax at this stage. BJ> 3) Modify the Python parser to allow optional static BJ> typing in the Python code, using the syntax developed in phase BJ> 1. Beef up the cross-checking program developed in phase 2 to BJ> use these type declarations to aid it in type inferencing. I'm not sure you need to go to #3. Why not keep the .pi files separate? Yes it's more work, but you also don't have to worry about the "optional" part of "optional static types". If the program could run just as well after removing all the .pi files, you're good to go. -Barry From tim_one@email.msn.com Wed Nov 25 22:54:59 1998 From: tim_one@email.msn.com (Tim Peters) Date: Wed, 25 Nov 1998 17:54:59 -0500 Subject: [Types-sig] Re: PRE-PROPOSAL: Python Interfaces In-Reply-To: <13915.20041.23787.219850@anthem.cnri.reston.va.us> Message-ID: <000501be18c6$a0bbb540$cb9e2299@tim> [TimP] > As we discussed on the egroups version of this, if interfaces > allow supplying impementation, then > > from InterfaceSupport import deferred > class Stack(Interface): > def push(self, thing): deferred() > etc > > would make everyone who *claims* to implement Stack pick up > the (presumably error-raising) Stack.push "deferred()" > implementation if they don't supply their own (or inherit some > other) real implementation of push. [BarryW] > That definitely does the trick, assuming we add something like > a NotImplementedError to the core and perhaps even a builtin > function deferred() that just raises the exception. Half of me > wants something a little more magical (and thus less dependent > on the author to play> the right game)... okay make that 1/4 of > me.. no 1/8... urg. 7/16'ths of me too, but I believe the groundrules for the Scarecrow Proposal include "no language changes". That leaves manual, or at best semi-automated, convention for now. In a Python2 world, I doubt it would be more of an insanely protracted battle than usual to get a "defer" keyword added, provided it worked out in practice by hand first. Given "def push(self, thing): defer" we could peek at the bytecode today, note the senseless LOAD_GLOBAL of "defer", and simply rewrite the user's code for them . illness-runs-in-the-family-ly y'rs - tim From Vladimir.Marangozov@inrialpes.fr Thu Nov 26 18:07:20 1998 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Thu, 26 Nov 1998 19:07:20 +0100 (NFT) Subject: [Types-sig] Type/class unification Message-ID: <199811261807.TAA29464@pukapuka.inrialpes.fr> I take advantage of the silence caused by the massive US turkey feast to drop a line regarding the type/class unification vs. the interface discussion. It seems to me that if the "clype" remake is MOPish enough, many people would feel more comfortable w.r.t. what they think a Python interface should look like. By definition, a MOP is like the statue of Liberty -- it says that we should feel free to do whatever we like ;-) A MOPish clype redesign should give us means to inspect or reshape every piece of code as a surgeon, so if someone wants to ask at some point whether an instance or a class conforms to some protocol/interface, all that has to be done is: - decide what is labeled as an interface / protocol / class - use the MOP to build a function, based on the previous decision, that will answer the question It's quite unclear to me, even after the 50 types-sig messages that landed in my inbox, what an (optional) Python interface is, and why should I feel happier if the Scarecrow (or another) formula is implemented in some way, given the actual implementation of Python's classes. Obviously, there's some need out there that I don't really understand and there are limitations in Python that prevent solving the problem easily. Anyway, I wonder whether if we have another implementation of classes and more tools to hammer on them (i.e inspect & modify their structure & behavior) the interface/static typing issues wouldn't become clearer (or better yet, irrelevant :-) ... An aside: An interesting pointer appeared in c.l.py about a language called Ruby which has some nice properties. IMHO it worths a look at the implementation. (Here's a comment from a file "object.c" that bootstraps the Ruby Universe: /* * Ruby's Class Hierarchy Chart * * +------------------+ * | | * Object---->(Object) | * ^ ^ ^ ^ | * | | | | | * | | +-----+ +---------+ | * | | | | | * | +-----------+ | | * | | | | | * +------+ | Module--->(Module) | * | | ^ ^ | * OtherClass-->(OtherClass) | | | * | | | * Class---->(Class) | * ^ | * | | * +----------------+ * * + All metaclasses are instances of the class `Class'. */ This reminds me of a diagram Guido posted on the obsolete egroups list, which contains another kind of animals (turtles). I'm appending Guido's message for the sake of the comparison, hoping that some of us would discuss the class/type unification. In my eyes, the practical solution of this problem is quite important and it may well influence the interface discussion. -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From Vladimir.Marangozov@inrialpes.fr Thu Nov 26 18:11:36 1998 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Thu, 26 Nov 1998 19:11:36 +0100 (NFT) Subject: [Types-sig] Re: Type/class unification In-Reply-To: from "marangoz" at "Nov 26, 98 07:07:20 pm" Message-ID: <199811261811.TAA21298@pukapuka.inrialpes.fr> Forgot to include Guido's masterpiece (and save you digging egroups). -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From: Guido van Rossum <@CNRI.Reston.VA.US> Subject: [python-classes] ASCII art Content-Type: text/plain Content-Transfer-Encoding: 7bit I've been working on a response for days now, and I'm giving up. Instead, I'll try to draw some stuff that I drew on a whiteboard today in front of a bunch of Norwegian and local Pythoneers. Consider instance x of class C, derived from class B. This is the ideal diagram: +-----------+ | x | +-----------+ | | v +-----------+ +-----------+ | C |---->| B | +-----------+ +-----------+ \ / \ / v v +-----------+ | class | +-----------+ I use the following convention: an arrow pointing down goes from an object to its meta-object, class or type. An arrow pointing right goes from a class to its base class. The idea is that you follow the down arrow to get to an object's behavior -- so x's behavior comes from C, and C's behavior comes from 'class' (and so does B's). You follow the arrow right to delegate attributes to a base class. (With multiple inheritance, you draw multipl arrows.) Note that the delegation from C to B will allow x to find behavior in B if it isn't found in C. In the current CPython implementation, things look a bit different: +-----------+ | x | +-----------+ / | / | / v / +-----------+ +-----------+ / | C |---->| B | / +-----------+ +-----------+ v \ / *===========* \ / I instance I v v *===========* *===========* \ I class I \ *===========* \ / \ / v v *===========* I type I *===========* That is, x has *two* behavior-defining links: one to the type 'instance', one to the class C. (I've drawn type objects in a different style here.) This is what causes a lot of grief. I'd like to move to the first diagram. It can be extended downwards and to the right. You could have as many levels of metaclasses as you want, and it would all be well defined: behavior comes from below, delegated behavior from the right. I've got to go soon, and I want this to go out tonight, so I'll just add some quick notes. - I like David's idea of specifying the metaclass in the class statement. Specifying the base class is specifying the arrow to the right; specifying the metaclass is specifying the arrow down. - There is a difference between class methods (which specify behavior of the class, and are defined in the metaclass) and C++/Java style static methods (which should be defined in the class using a special syntax so that they don't belong to a specific instance; however, as in C++/Java, it should still be *possible* to invoke them via the instance, as well as via the class.) [I had a longer rant about this, but no time to condense it now.] - There could be a class "Object" which is used as the default right arrow. - There could also be a class "Turtles-all-the-way-down" which is used as the default arrow down. - I don't know if Object and Turtles are the same object. I know that each has a right arrow to Object and each has a down arrow to Turtles; they *could* be related like this: ------- --------- ------------------ | | | \ | | | v v \ | | | +---------+ \ \->+--------+ | | | Turtles |------------->| Object |---/ | +---------+ \ +--------+ | | \ | \ / \ / ---------- -------- (Heh! :-) - There are plenty of problems left. Later, after my drumming class :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From Justus Pendleton Fri Nov 27 06:08:24 1998 From: Justus Pendleton (Justus Pendleton) Date: Fri, 27 Nov 1998 01:08:24 -0500 Subject: [Types-sig] Why I don't like static types in Python In-Reply-To: <199811261807.TAA29464@pukapuka.inrialpes.fr>; from Vladimir Marangozov on Thu, Nov 26, 1998 at 07:07:20PM +0100 References: <199811261807.TAA29464@pukapuka.inrialpes.fr> Message-ID: <19981127010824.A488@ryoohki> On Thu, Nov 26, 1998 at 07:07:20PM +0100, Vladimir Marangozov wrote: > It's quite unclear to me, even after the 50 types-sig messages that landed > in my inbox, what an (optional) Python interface is, and why should I feel > happier if the Scarecrow (or another) formula is implemented in some way I must say that I am in much the same state. I haven't really seen anyone try to explain why I should feel happier if static types and interfaces are implemented in Python. Maybe if someone could give me an example of a real world problem they couldn't solve in Python because it lacked these features it would be easier for me to understand the need we are trying to address here. Having read through Roger Masse's original paper on "Add Later" types as well as the various follow ups it seems that the reasons to use static types are: 1. Static types can improve readability by making the programmer's intentions more explicit, 2. Optimize for increased performance, 3. For detecting type errors, 4. The existing type system in Python is nice for development within small groups but is not well suited for large-scale development because of the difficulty in specifying a common interface without implementation to convey behavior. All in all, those are, in my opinion, thoroughly unconvincing arguments. [Caveat: I am neither a language nor a compiler designer and don't know a whole lot about either so I may be talking out of my ass on all of this. If I am, I'm sure someone will let me know :-) ] ------------------------------------------------------------------------- 1. I don't think that static types really improve readability in any way whatsoever. The problem? The type is only shown when the variable is declared, not every time it is used. Even though C is statically typed lots of people use Hungarian notation to get around this very problem. If we are really interested in increasing the readability of programs by making the programmer's intentions more explicit then why don't we develop a Dutch naming scheme and use that (i.e. port_h [h => heel]) instead? ------------------------------------------------------------------------- 2. I agree with Skip Montinaro's comments that it's not clear we've come close to exhausting the performance possibilities of Python as it currently exists. He and others have pointed out other interesting avenues to explore. If performance is the concern then I think rather than changing the syntax in any way we should first try to make nonvisible changes. Maybe there could be some kind of JIT compiler for Python, I dunno. Would it be possible for there to be a Python compiler that does global analysis of the entire program and then does some magic type inferencing and dynamic code recompilation to notice that var is only ever assigned integers and then generate appropriate byte code for that situation? What's more, I'm not convinced that performance is such an issue. I'm sure that other people have had other experiences, but at my work the only times that speed has ever mattered it has mattered so much that Python could never possibly be a contender no matter how fast it is. When Python is playing the role of a glue or an extension language, it's already fast enough for every use I've ever seen. On the other hand, even if we do nothing Python will be twice as fast next year as it is this year. How fast is fast enough? How slow is too slow? More to the point, how much will static typing in Python actually affect these things? Would we be willing to add a change as major as static typing for a 10% speed increase? ------------------------------------------------------------------------- 3. Roger Masse writes that static typing provides an 'improved level of type safety and program correctness' but I'm not aware of any empirical evidence of this. Is this just simply "common sense"? I am wary of "common sense" that is lacking an empirical foundation. Don't forget that for many people it was "common sense" that the earth was flat, that the earth was the center of the universe, and that maggots spontaneously generated from rotten meat. Why should we rush to change our favorite language when their is no proof that doing so will provide the benefits we claim we want? Is a strongly typed system actually safer in any way whatsoever? As Dave Beazley asks, "are there really huge numbers of people out there writing unreliable Python programs? Is type-safety going to solve their problem even if it were available?" Why add a language feature that doesn't solve a problem? C is a weakly typed language but people still seem to be able to create "safe" systems in it.... IMHO there would be some problems with static typing in Python anyway. Would the static type system catch something like the following: ==================================== def myCallable( d : MyClass): return d.my_method() val : MyClass val = MyClass() del val.my_method myCallable(val) ==================================== Would the type system notice that I am turning val into something that no longer conforms to the MyClass interface? In any case, the static typing unnecessarily restricts me in this example. The function 'myCallable' doesn't really need to have an object of MyClass passed in. It only needs something that has the 'my_method'. The proposal mentions COMPARABLE as one possible 'protocol' used in static typing. But what if I only implement < and >? I would still be able to use a lot of methods that claim to only take COMPARABLE...but the typing system would prevent me from doing so. Say some library designer decides that his function will only take numbers. Now, he _thinks_ that his function only works on integers. Actually, he's wrong. Because I have this nifty class that I designed that mostly works like numbers and his function would product the right result if it were called on my class. But my class aren't numbers. Maybe my class doesn't form an abelian group under addition for whatever reason. But that has no effect whatsoever on the ability of this one function to Do The Right Thing to my class. But I can't use it because the library writer _thought_ he knew everything about every future user of his functions. Roger Masse asks of dynamic typing, "More importantly, how do you gain confidence that an implementation of a file-like object has implemented seek fully without running it?" Just because I am using a statically typed language I am not saved. Sure I may have a file-like object that has a method named seek that takes the parameters I think it's supposed to. But how do I know it doesn't just delete the file handle instead of actually doing a seek? With a statically typed language Masse's question simply becomes, "More importantly, how do you gain confidence that an implementation of a file-like object has implemented seek in the way you think it ought to implement seek without running it?" In short, I don't think that static typing gives you a whole lot in terms of safety...certainly not enough to warrant adding the feature to the core language, even as an optional one. If we had Design By Contract in Python (with inherited assertions and so forth) would that be sufficient to satisfy our needs for safety? [BTW, Roger Masse writes that "static typing is a prerequisite for design by contract" but I don't understand why.] ------------------------------------------------------------------------- 4. I agree that specifying an interface without the implementation would be nice for larger projects. But couldn't that same kind of information be generated by some documentation tool like interdoc or whatever the docutils people have cooked up? Shouldn't people be reading the documentation for the code rather than the code anyway? Why would we want to encourage people to bypass the documentation and read the code directly? Why would we want to encourage skimping on documentation? It seems that adding static typing to Python as a way of providing documentation to Python code is somewhat misguided. Instead shouldn't we use __doc__ strings with structured formatting? ------------------------------------------------------------------------- In short my complaints about static typing (and protocols/interfaces, I suppose) are that I don't see what problems these new features would solve that currently available features can't. I think there are definitely ways that Python can improve and grow, I just don't think that static types are one of those ways. I look forward to hearing what proponents of static typing and protocols think about this. -- Justus Pendleton From fredrik@pythonware.com Fri Nov 27 09:21:40 1998 From: fredrik@pythonware.com (Fredrik Lundh) Date: Fri, 27 Nov 1998 10:21:40 +0100 Subject: [Types-sig] Why I don't like static types in Python Message-ID: <00fc01be19e7$59fe9970$f29b12c2@pythonware.com> Justus Pendleton wrote: >I must say that I am in much the same state. I haven't really seen anyone >try to explain why I should feel happier if static types and interfaces are >implemented in Python. Hmm. Based on what's said on http://www.python.org/sigs/types-sig/ I thought this was a working group that was going to refine and possibly also work on implementing the various typing issues that were discussed by many highly experienced Python developers in several sessions at the recent Python conference, not a general discussion group. Have you read that page and come to another conclusion? (If thinking about typed Python makes you feel uneasy, why not ignore the whole thing? I mean, nobody has suggested that typing should be mandatory in future versions of Python...) >Would it be possible for there to be a Python compiler that does global >analysis of the entire program and then does some magic type inferencing >and dynamic code recompilation to notice that var is only ever assigned >integers and then generate appropriate byte code for that situation? Yes. There are such compilers today. But in a world where all larger programs are dynamically configured, global analysis is a relatively worthless concept. >What's more, I'm not convinced that performance is such an issue. >I'm sure that other people have had other experiences Correct. Other people have other experiences. >More to the point, how much will static typing in Python actually affect >these things? Would we be willing to add a change as major as static >typing for a 10% speed increase? We're not talking 10% speed increase. We're talking near-C speed. And from another perspective, much lower development costs for equivalent code. >Roger Masse writes that static typing provides an 'improved level of type >safety and program correctness' but I'm not aware of any empirical evidence >of this. How large is your largest Python application? 10k lines? 20k lines? 50k lines? How many people worked on your development team? How many design iterations have you had this far? >Maybe if someone could give me an example of a real world problem >they couldn't solve in Python because it lacked these features it would >be easier for me to understand the need we are trying to address here. http://www.python.org/sigs/types-sig/ Note that there are *three* different issues at hand. Static typing is just one of them. /F From Justus Pendleton Fri Nov 27 17:08:52 1998 From: Justus Pendleton (Justus Pendleton) Date: Fri, 27 Nov 1998 12:08:52 -0500 Subject: [Types-sig] Why I don't like static types in Python In-Reply-To: <00fc01be19e7$59fe9970$f29b12c2@pythonware.com>; from Fredrik Lundh on Fri, Nov 27, 1998 at 10:21:40AM +0100 References: <00fc01be19e7$59fe9970$f29b12c2@pythonware.com> Message-ID: <19981127120852.B439@ryoohki> On Fri, Nov 27, 1998 at 10:21:40AM +0100, Fredrik Lundh wrote: > Hmm. Based on what's said on http://www.python.org/sigs/types-sig/ > I thought this was a working group that was going to refine and possibly > also work on implementing the various typing issues that were discussed > by many highly experienced Python developers in several sessions at the > recent Python conference, not a general discussion group. Have you read > that page and come to another conclusion? Hmmm...the page I read said "This list has been created for discussions on issues related to Python's type system." When I click on the link for "Static Typing Proposals" it says "coming soon". When I click on the link for "Interface definitions" it says "coming soon". My response was based on reading Roger Masse's current position paper, "Add Later Static Types for Python" and his summary of the developer day session on the same topic. As far as I can tell this is a working group for "discussions on issues related to Python's type system" and I think that before anyone starts discussing how to implement static types it might be good if it were clear what the goal of such a system might be. I couldn't find a clear description of that goal in anything that I read. My post was an attempt to clear up what that goal might be. Perhaps some of these "many highly experienced Python developers" could answer my questions without resorting to the Authority or Social Proof methods of persuasion? I don't care how experienced the doctor is, I usual prefer an explanation of why he needs to conduct a rectal examination.... > But in a world where all larger programs are dynamically configured, global > analysis is a relatively worthless concept. Out of curiosity (since I don't know) does this mean that run-time and/or dynamic recompilation are equally fruitless avenues of inquiry? > We're not talking 10% speed increase. We're talking near-C speed. > And from another perspective, much lower development costs for > equivalent code. I admit that I am skeptical of claims of near-C speed :-) Java is statically typed and isn't really "near-C speed". Python claims to be a higher level language than Java so I have difficulty believing it will succeed where Java fails....especially since Python's static typing would be voluntary which makes me believe that it wouldn't be as effective as Java's mandatory static typing. Maybe if someone had a working example of a statically typed, dynamic, higher level, object oriented language that has near-C speed I would be more easily convinced..? I have a question (since I don't know), if the static typing is voluntary then does that mean that nothing in the core python library could use static typing? > >Roger Masse writes that static typing provides an 'improved level of type > >safety and program correctness' but I'm not aware of any empirical evidence > >of this. > > How large is your largest Python application? 10k lines? 20k lines? > 50k lines? How many people worked on your development team? > How many design iterations have you had this far? Hmmm...this doesn't really answer my concern about the lack of empirical evidence of the usefulness of static typing. Anecdotal evidence simply isn't enough as far as I'm concerned. For years there was anecdotal evidence that client/server systems were the brilliant wave of the future, but in the November/December issue of IEEE Software Diomidis Spinelli argues convincingly that client/server systems have failed. > >Maybe if someone could give me an example of a real world problem > >they couldn't solve in Python because it lacked these features it would > >be easier for me to understand the need we are trying to address here. > > http://www.python.org/sigs/types-sig/ I am subscribed to the list and have read every message that has ever been posted to the list (all 54 of them :) Virtually all of them are about interfaces. Virtually all of them are about implementation. If you could actually point out which one of those 54 messages contain "an example of a real world problem they couldn't solve in Python because it lacked these features it would be easier for me to understand the need we are trying to address here." > Note that there are *three* different issues at hand. Static typing > is just one of them. I realize that and my post mixed together the issues somewhat :-(, I apologize for that. -- Justus Pendleton From dominic.binks@aethos.co.uk Fri Nov 27 17:52:35 1998 From: dominic.binks@aethos.co.uk (Dominic Binks) Date: Fri, 27 Nov 1998 17:52:35 +0000 Subject: [Types-sig] Why I don't like static types in Python References: <199811271701.MAA00988@python.org> Message-ID: <365EE6E3.CF4C91B9@aethos.co.uk> Justus Pendleton writes: > > All in all, those are, in my opinion, thoroughly unconvincing arguments. > > 1. Static types can improve readability by making the programmer's > intentions more explicit, ---- > 1. I don't think that static types really improve readability in any way > whatsoever. The problem? The type is only shown when the variable is > declared, not every time it is used. Even though C is statically typed lots > of people use Hungarian notation to get around this very problem. If we are > really interested in increasing the readability of programs by making the > programmer's intentions more explicit then why don't we develop a Dutch > naming scheme and use that (i.e. port_h [h => heel]) instead? True, but if types are in a kind of prototype, you know where you can look for the order of arguments, what return type is expected etc. As a programmer, trying coding C without header files declare prototypes. Also trying making sense of C function man pages without type information. This information provides instructions on what should be provided as arguments. In C you can get away with (as far as the compiler is concerned) sticking almost anything in the arguments to a function, but the prototype tells you if you stick wrongly typed arguments into the functions you are at very best relying on undefined behaviour and at worst you are shooting yourself in the foot. Python is slightly different in this respect because portability is not an immediate problem, since the interpreter takes care of it, but I have to look up the manual to find out the order of arguments for any function I wish to use. A prototype would save me this. I know you can read the code, but this is not in general a valid answer to the problem. > 2. Optimize for increased performance, > ------------------------------------------------------------------------- > 2. I agree with Skip Montinaro's comments that it's not clear we've come > close to exhausting the performance possibilities of Python as it currently > exists. He and others have pointed out other interesting avenues to explore. > If performance is the concern then I think rather than changing the syntax in > any way we should first try to make nonvisible changes. Maybe there could be > some kind of JIT compiler for Python, I dunno. Would it be possible for > there to be a Python compiler that does global analysis of the entire program > and then does some magic type inferencing and dynamic code recompilation to > notice that var is only ever assigned integers and then generate appropriate > byte code for that situation? > > What's more, I'm not convinced that performance is such an issue. I'm > sure that other people have had other experiences, but at my work the only > times that speed has ever mattered it has mattered so much that Python could > never possibly be a contender no matter how fast it is. When Python is > playing the role of a glue or an extension language, it's already fast enough > for every use I've ever seen. On the other hand, even if we do nothing > Python will be twice as fast next year as it is this year. How fast is fast > enough? How slow is too slow? > > More to the point, how much will static typing in Python actually affect > these things? Would we be willing to add a change as major as static typing > for a 10% speed increase? This is all true but I think in tight loops you see much more than a 10% increase in performance. The tricks that Skip describes in his optimisation hints include precisely the kinds of optimisations that a compiler which was type aware could perform automatically: i.e. avoid dictionary lookups for names which is the single most important aspect of enhancing the performance of a Python application (assuming the algorithm is good!). > 3. For detecting type errors, > ------------------------------------------------------------------------- > 3. Roger Masse writes that static typing provides an 'improved level of type > safety and program correctness' but I'm not aware of any empirical evidence > of this. Is this just simply "common sense"? I am wary of "common sense" > that is lacking an empirical foundation. Don't forget that for many people > it was "common sense" that the earth was flat, that the earth was the center > of the universe, and that maggots spontaneously generated from rotten meat. > Why should we rush to change our favorite language when their is no proof > that doing so will provide the benefits we claim we want? > > Is a strongly typed system actually safer in any way whatsoever? As Dave > Beazley asks, "are there really huge numbers of people out there writing > unreliable Python programs? Is type-safety going to solve their problem even > if it were available?" Why add a language feature that doesn't solve a > problem? C is a weakly typed language but people still seem to be able to > create "safe" systems in it.... > > IMHO there would be some problems with static typing in Python anyway. Would > the static type system catch something like the following: > > ==================================== > def myCallable( d : MyClass): > return d.my_method() > > val : MyClass > val = MyClass() > del val.my_method > myCallable(val) > ==================================== To use an similar example from C: struct my_struct { char *my_string; }; ... struct my_struct val; val.my_string = strdup("Hello World!"); free(val.my_string); printf("%s\n", val.my_string); Ok it's not a function, but the basic argument is the same. Runtime ordering of operations cannot obviously be detected by the type system. It isn't there to do that. It is there to try to assist the programmer in detecting areas where they've (inadvertantly) passed the wrong argument to a function/method. [Further more reasonable arguments removed] > > "More importantly, how do you gain confidence that an implementation of a > file-like object has implemented seek in the way you think it ought to > implement seek without running it?" So the question ends up does it not make sense that two methods in two objects that have the same name do the same thing using the same arguments. Surely it makes sense if seek make sense in my file like object it makes sense that it takes the same arguments as the seek in the file object (meaning that I can replace the file object with my file-like object). > > In short, I don't think that static typing gives you a whole lot in terms of > safety...certainly not enough to warrant adding the feature to the core > language, even as an optional one. If we had Design By Contract in Python > (with inherited assertions and so forth) would that be sufficient to satisfy > our needs for safety? A bit of background about what we use Python for. We use it as a kind of glue, kind of RAD tool. It provides an interface for other systems to add/query/modify/delete customers to our system. It allows us great flexibility and ease of fixing problems, new featiures etc. It takes about a day to provide support for a new operation (i.e. add or query) and about week to debug it. Probably about half of that can be placed at easily checkable static conditions, type correctness being one of the most important (usually because it's often hard to locate). I have suffered because of Python's lack of type system and yet even a primitive system that says this function should take a string and you've given it an integer would have saved me and my team many hours. Finally I'd like to add that it may be impossible to do something that covers the whole of Python. eval() for example presents some real problems for any type system to provide any real safety. However what can be done can save some valuable amount of development time. > 4. The existing type system in Python is nice for development within small > groups but is not well suited for large-scale development because of the > difficulty in specifying a common interface without implementation to > convey behavior. ------------------------------------------------------------------------- > 4. I agree that specifying an interface without the implementation would be > nice for larger projects. But couldn't that same kind of information be > generated by some documentation tool like interdoc or whatever the docutils > people have cooked up? Yes, but how can you tell what type the arguments of a function/class should be without the type system. Yes you can use the type checks that Python allows you to do, but that means much extra complexity in the autodoc tools. The prototypes would be additional the document strings. > Shouldn't people be reading the documentation for the > code rather than the code anyway? Why would we want to encourage people to > bypass the documentation and read the code directly? Why would we want to > encourage skimping on documentation? It seems that adding static typing to > Python as a way of providing documentation to Python code is somewhat > misguided. Instead shouldn't we use __doc__ strings with structured > formatting? > But that still won't guard against putting an integer into a function/method that is expecting a string. > ------------------------------------------------------------------------- > In short my complaints about static typing (and protocols/interfaces, I > suppose) are that I don't see what problems these new features would solve > that currently available features can't. Probably nothing. It's more a question of ease of use. It makes much more sense to provide an interface to check types rather than at runtime (reduced overhead in time) and implementors will not always do what they should do. Providing an interface ensures that they will at least instruct a compiler to do the checks. > I think there are definitely ways > that Python can improve and grow, I just don't think that static types are > one of those ways. I look forward to hearing what proponents of static > typing and protocols think about this. > I'd love to see type inference. I know that seems to be a dirty word around the Python community but it certainly works really well in functional languages and costs nothing t implement for implementors. For Python it could easily be just a tool, that way you don't use it if you don't want to. Just my 2p Dominic -- Dominic Binks/Systems Engineer/Aethos Communication Systems Ltd. 400 Park Avenue/Aztec West/Bristol/BS32 4TR/United Kingdom Tel: +44 1454 614455/Fax: +44 1454 620527/Mobile: +44 498 693964 E-mail: dominic.binks@aethos.co.uk From gmcm@hypernet.com Fri Nov 27 18:40:39 1998 From: gmcm@hypernet.com (Gordon McMillan) Date: Fri, 27 Nov 1998 13:40:39 -0500 Subject: [Types-sig] Why I don't like static types in Python In-Reply-To: <19981127120852.B439@ryoohki> References: <00fc01be19e7$59fe9970$f29b12c2@pythonware.com>; from Fredrik Lundh on Fri, Nov 27, 1998 at 10:21:40AM +0100 Message-ID: <1299947911-634177@hypernet.com> Justus Pendleton wrote: > On Fri, Nov 27, 1998 at 10:21:40AM +0100, Fredrik Lundh wrote: > > Hmm. Based on what's said on http://www.python.org/sigs/types-sig/ [snip] > When I click on the link for "Static Typing Proposals" it says > "coming soon". When I click on the link for "Interface definitions" > it says "coming soon". The summaries are now there: http://www.foretec.com/python/workshops/1998-11/devday.html > My response was based on reading Roger Masse's current position > paper, "Add Later Static Types for Python" and his summary of the > developer day session on the same topic. > > As far as I can tell this is a working group for "discussions on > issues related to Python's type system" and I think that before > anyone starts discussing how to implement static types it might be > good if it were clear what the goal of such a system might be. I > couldn't find a clear description of that goal in anything that I > read. My post was an attempt to clear up what that goal might be. Unfortunately, there are nearly as many goals as there are interested developers. And some of those goals are "to see if turns into something interesting", which is damned hard to quantify. One of my (stated) goals is to make sure nothing undesirable gets rammed down my throat. You could assist in that regard, but not by saying that people shouldn't be interested, or shouldn't be playing with possible implementations. > > But in a world where all larger programs are dynamically configured, global > > analysis is a relatively worthless concept. Now Fredrik, it's not worthless (or even "relatively worthless"), just of limited applicability. And you can certainly learn things from the exercise that might help with local program analysis. > Out of curiosity (since I don't know) does this mean that run-time > and/or dynamic recompilation are equally fruitless avenues of > inquiry? They're a lot harder. For one thing, you have to optimize the analysis. > > We're not talking 10% speed increase. We're talking near-C speed. > > And from another perspective, much lower development costs for > > equivalent code. > > I admit that I am skeptical of claims of near-C speed :-) Java is > statically typed and isn't really "near-C speed". Python claims to > be a higher level language than Java so I have difficulty believing > it will succeed where Java fails....especially since Python's static > typing would be voluntary which makes me believe that it wouldn't be > as effective as Java's mandatory static typing. Jim Hugunin used global program analysis to demonstrate (admittedly contrived examples of) JPythonc compiled code running 2000X faster than CPython. In the example, his analysis was able to arrive at the conclusion that a Python Int could actually be compiled to the Java for a native int. > I have a question (since I don't know), if the static typing is > voluntary then does that mean that nothing in the core python > library could use static typing? Python 1.6 is intended to be as backward compatible as possible. Whatever pieces of these 3 topics show up in 1.6 should be safely ignorable if that's what you want. Python 2.0 (a long way off) is another story. > Hmmm...this doesn't really answer my concern about the lack of > empirical evidence of the usefulness of static typing. Anecdotal > evidence simply isn't enough as far as I'm concerned. For years > there was anecdotal evidence that client/server systems were the > brilliant wave of the future, but in the November/December issue of > IEEE Software Diomidis Spinelli argues convincingly that > client/server systems have failed. I'm not aware of any empirical evidence saying that empirical evidence should be a prerequisite for exploring new language features. On the contrary, there is empirical evidence that empirical evidence can be found for pretty much any conclusion desired. Without having read it, I would be tempted to site your citation as an example . Just go through any out-of-date papers from any scientific discipline. There are plenty of examples of studies with empirical evidence that today are completely discredited. Pick up Against Method by Paul K. Feyerabend. http://www.amazon.com/exec/obidos/ASIN/0860916464/r/002-8722431-097041 2 > > >Maybe if someone could give me an example of a real world problem > > >they couldn't solve in Python because it lacked these features it would > > >be easier for me to understand the need we are trying to address here. Very rarely is anything about "couldn't". It's about making it easier or more natural or faster. I think if you want to see where Jim Fulton and the "interfaces" proposal is coming from, you should look at Bobo, where he uses introspection heavily so that the user can write very natural Python, but Bobo can determine what is "safe" to publish as CGI. There's plenty of empirical evidence that Python is "slow", which is constantly thrown in our faces, despite the fact that it is usually very misleading. It's also quite clear that static typing can prevent a kind of error that is very difficult to prove absent in a dynamic system. Whether it's worthwhile is another question, and one that really isn't answered for Python by looking at other languages. Unfortunately, it looks like the type / class thing can't be resolved until Python 2. But I'd be surprised if anyone asked for empirical evidence that this is a flaw in the language. - Gordon From Vladimir.Marangozov@inrialpes.fr Fri Nov 27 21:15:57 1998 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Fri, 27 Nov 1998 22:15:57 +0100 (NFT) Subject: [Types-sig] Why I don't like static types in Python In-Reply-To: <1299947911-634177@hypernet.com> from "Gordon McMillan" at "Nov 27, 98 01:40:39 pm" Message-ID: <199811272115.WAA28894@pukapuka.inrialpes.fr> Gordon McMillan wrote: > > Python 1.6 is intended to be as backward compatible as possible. > Whatever pieces of these 3 topics show up in 1.6 should be safely > ignorable if that's what you want. > > Python 2.0 (a long way off) is another story. Hm. I don't mind start prototyping bits of Python 2 alpha today. This is a major effort and it may well take more time than expected. Especially when the type/class problem is already identified. Most of us agree that it should be solved, and it is known that this would be an incompatible change with the 1.x line. The new class model would be at the heart of Python 2, that's why I tend to give priority to its discussion. In the meantime, any 'short term' proposal for an interface system is welcome, of course. However, I suspect that things may change dramatically in Py 2. JimF mentioned that his 1st implementation of the Scarecrow proposal is based on the metaclass hook. I don't object to this, but I see the approach as yet another sign that: -- the current class implementation is inappropriate for some tasks -- sometimes, people want more control over Python's internals, and since they don't have it today, they consider the metaclass hook. I don't think this is good practice, although I understand that the hook may be convenient for alleviating the lack of control. -- From what I understand of the interface problem, I deduce that part of it is due to the current state of affairs about Python's classes. And in this sense precisely, ** given the actual implementation ** I said that I don't really understand how we can remedy the interface problem and its (static-types, optimization, etc.) derivatives. (Justus Pendleton quoted me partially on this). -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From Justus Pendleton Fri Nov 27 21:48:41 1998 From: Justus Pendleton (Justus Pendleton) Date: Fri, 27 Nov 1998 16:48:41 -0500 Subject: [Types-sig] Why I don't like static types in Python In-Reply-To: <365EE6E3.CF4C91B9@aethos.co.uk>; from Dominic Binks on Fri, Nov 27, 1998 at 05:52:35PM +0000 References: <199811271701.MAA00988@python.org> <365EE6E3.CF4C91B9@aethos.co.uk> Message-ID: <19981127164841.A1370@ryoohki> On Fri, Nov 27, 1998 at 05:52:35PM +0000, Dominic Binks wrote: > but I have to look up the manual to find out the order of arguments for any > function I wish to use. A prototype would save me this. I know you can > read the code, but this is not in general a valid answer to the problem. Hmmm...I'm not sure what the difference between looking up the parameters in a manual versus looking up the parameters in a prototype are. Seems like they are exactly the same thing except that the manual is far more likely to actually convey some useful information...like what the hell "int *decpt" :-) > To use an similar example from C: > > struct my_struct { > char *my_string; > }; > > .. > > struct my_struct val; > > val.my_string = strdup("Hello World!"); > > free(val.my_string); > > printf("%s\n", val.my_string); > > Ok it's not a function, but the basic argument is the same. Runtime > ordering of operations cannot obviously be detected by the type system. > It isn't there to do that. It is there to try to assist the programmer > in detecting areas where they've (inadvertantly) passed the wrong > argument to a function/method. I would argue that the above code simply shows that C is broken and nothing more :-). The parameter to printf is a STRING_OBJECT, but since C is a hack it calls it a char * instead. When you do free on a STRING_OBJECT it is no longer a STRING_OBJECT. If I do x : Int x = 5 del x get_type_of(x) Why isn't x still an Int? Would a type system catch this error: def foo(x : String): .... x = 'tree' x = 1 foo(x) I'm confused about why a type system would catch this but not my original example...especially if ordering of operations cannot be detected by the type system. > A bit of background about what we use Python for. Thanks, this is exactly the kind of background I was hoping someone would share to illuminate and educate me ;-) > Probably about half of that can be placed at easily checkable static > conditions, type correctness being one of the most important (usually > because it's often hard to locate). If Python had some kind of type inferencing engine would it be possible to have the compiler give a warning at compile time that it looks like you might be doing something wrong ... without the programmer necessarily having to add explicit typing information to the code? -- Justus Pendleton From tim_one@email.msn.com Sat Nov 28 02:20:51 1998 From: tim_one@email.msn.com (Tim Peters) Date: Fri, 27 Nov 1998 21:20:51 -0500 Subject: [Types-sig] Why I don't like static types in Python In-Reply-To: <1299947911-634177@hypernet.com> Message-ID: <000f01be1a75$b80b59c0$6d9e2299@tim> [Gordon McMillan] > ... > Jim Hugunin used global program analysis to demonstrate > (admittedly contrived examples of) JPythonc compiled code > running 2000X faster than CPython. In the example, his > analysis was able to arrive at the conclusion that a Python > Int could actually be compiled to the Java for a native int. FYI, at work we've seen Java recodings of our core computational kernels reach 80% of optimized C++ speed, using a now somewhat out-of-date version of Symantec's JIT. That isn't fast enough, and neither is optimized C++ (our actual kernels are hand-tweaked assembler), but it is a sign of the way things are going. [Justus Pendleton] > ... > I have a question (since I don't know), if the static typing is > voluntary then does that mean that nothing in the core python > library could use static typing? Of course not. If e.g. all-integer lists ran 100x faster, you might see an IntList.py added to the core, and maybe you'd be required to use static typing *if* you wanted to use that module. There are no consequence-free changes. [Gordon] > ... > It's also quite clear that static typing can prevent a kind of error > that is very difficult to prove absent in a dynamic system. Whether > it's worthwhile is another question, and one that really isn't > answered for Python by looking at other languages. I don't much care about Python's speed myself, and have been known to let a Python program run non-stop for weeks. It is the mother of all bitches, though, when at the end of the month it dies with an AttributeError because I typed: x.printReslut() instead of x.printResult() at the end. So I try not to do that . For some kinds of programs, I would be happy to declare, define, delimit and constrain every name in my program in triplicate, in return for a guarantee stuff like that wouldn't happen at runtime. > Unfortunately, it looks like the type / class thing can't be > resolved until Python 2. Unfortunate indeed! That's the most important issue on this SIG's table. > But I'd be surprised if anyone asked for empirical evidence that > this is a flaw in the language. I wouldn't . throw-in-some-empirical-evidence-that-programming- of-any-kind-makes-the-world-better-and-back-it- up-with-a-double-blind-experiment-ly y'rs - tim From tim_one@email.msn.com Sat Nov 28 02:20:49 1998 From: tim_one@email.msn.com (Tim Peters) Date: Fri, 27 Nov 1998 21:20:49 -0500 Subject: [Types-sig] Why I don't like static types in Python In-Reply-To: <365EE6E3.CF4C91B9@aethos.co.uk> Message-ID: <000e01be1a75$b6f5a400$6d9e2299@tim> [Dominic Binks] > ... > I'd love to see type inference. I know that seems to be a dirty > word around the Python community Not dirty, just harder in Python than you apparently believe. If you were to, say, offhandedly suggest that Python "should do" this and that it's no trouble at all, the glaring reaction you get should probably be taken more personally than technically . > but it certainly works really well in functional languages Ahem -- functional languages deliberately constrain their type systems to make type inference tractable. Python doesn't. > and costs nothing t implement for implementors. Well, in *that* case ... if-it-costs-nothing-to-implement-it-must-already-be-in-ly y'rs - tim From tim_one@email.msn.com Sat Nov 28 10:01:13 1998 From: tim_one@email.msn.com (Tim Peters) Date: Sat, 28 Nov 1998 05:01:13 -0500 Subject: [Types-sig] Why I don't like static types in Python In-Reply-To: <19981127164841.A1370@ryoohki> Message-ID: <000001be1ab6$08270ea0$659e2299@tim> [Justus Pendleton] > Hmmm...I'm not sure what the difference between looking up the > parameters in a manual versus looking up the parameters in a > prototype are. Seems like they are exactly the same thing > except that the manual is far more likely to actually convey > some useful information...like what the hell "int *decpt" :-) The most obvious difference is that a type mismatch between formal arguments in a prototype and actual arguments in a call can yield a compile-time error instead of (in C before prototypes) runtime insanity or (in Python) runtime TypeErrors. If you're looking for empirical evidence that this can be useful, next time you get a compile-time type error from C or C++, mindlessly cast the offending expression(s) to the type(s) it expected and time how long it takes a colleague to track down the resulting runtime problems <0.3 wink>. It doesn't matter what "int *decpt" is intended to mean if you pass it an int by mistake; static type systems catch *gross* errors, & it's always surprised me how many of those there are in a large project over time. > ... > Would a type system catch this error: > > def foo(x : String): > .... > x = 'tree' > x = 1 > foo(x) Not at compile-time, without adding flow-based type-inferencing too. OTOH, also declare x as being of type String, and it could bitch about the "x = 1" at compile-time (the real source of the error, not the call to "foo"). > ... > If Python had some kind of type inferencing engine would it be > possible to have the compiler give a warning at compile time > that it looks like you might be doing something wrong ... without > the programmer necessarily having to add explicit typing > information to the code? Yes, but that's an awfully big "if", and a type inferencer in Python likely can't get far without additional annotations either. For example, in import Utils x = Utils.Container() for i in 1, 2, 3: x.append(Utils.upgrade(i)) print i the class of x may change on every iteration of the loop -- and so may the object invoked by Utils.upgrade, and so may the object bound to Utils itself! (A call to Utils.upgrade may reach back into the module and change the module's binding for "Utils"). Without additional info, we can't even tell where Utils first comes from: the behavior of import can be affected by runtime sys.path fiddling -- or perhaps before the import some other chunk of code sucks Utils.py down from the Web and replaces the libs/Utils.py that existed at compile-time. The good news is that we're sure i is an int *some* of the time -- but the act of invoking Utils.__getattr__("upgrade") may reach back into the module and change the binding of "i" before the call to Utils.upgrade gets made. In particular, we have no idea what the type of "i" may be *after* the loop (int is a darned good guess, but the language doesn't guarantee it). An automagic system that can deal with all that is unlikely. An optional static type system that can deal with all that is, well, a bit less unlikely <0.8 wink>. Could be in the end that we just get a gimmick for automating more type checks to catch problems earlier. static-typing-*can*-pay-for-itself-even-if-it-slows- code-down-...-for-some-people-and-some-apps-ly y'rs - tim From billtut@microsoft.com Sun Nov 29 01:12:17 1998 From: billtut@microsoft.com (Bill Tutt) Date: Sat, 28 Nov 1998 17:12:17 -0800 Subject: [Types-sig] Why I don't like static types in Python Message-ID: <4D0A23B3F74DD111ACCD00805F31D8100DB90740@RED-MSG-50> > From: Tim Peters [mailto:tim_one@email.msn.com] > > Ahem -- functional languages deliberately constrain their > type systems to > make type inference tractable. Python doesn't. > Well, Self had a type infrencing engine written for it, and its argubably as dynamic, or more dynamic than Python. (Self has real closures, and no classes but does have inheritance). Ole Agesen's Ph.D. thesis (http://www.sunlabs.com/technical-reports/1996/abstract-52.html) "Concrete Type Inferencing: Delivering Object-Oriented Applications". However several factors make type infrencing Python fairly difficult: Type infrencing is best used in a closed environment. (akin to using freeze) The type infrencing system also benefits greatly from having detailed type signatures of C extensions. Which isn't to say it isn't possible, it is, its just complex and time consuming to implement. Anybody need a masters thesis topic? ;) Bill From Justus Pendleton Sun Nov 29 16:46:34 1998 From: Justus Pendleton (Justus Pendleton) Date: Sun, 29 Nov 1998 11:46:34 -0500 Subject: [Types-sig] Why I don't like static types in Python In-Reply-To: <000f01be1a75$b80b59c0$6d9e2299@tim>; from Tim Peters on Fri, Nov 27, 1998 at 09:20:51PM -0500 References: <1299947911-634177@hypernet.com> <000f01be1a75$b80b59c0$6d9e2299@tim> Message-ID: <19981129114634.B354@ryoohki> On Fri, Nov 27, 1998 at 09:20:51PM -0500, Tim Peters wrote: > > I have a question (since I don't know), if the static typing is > > voluntary then does that mean that nothing in the core python > > library could use static typing? > > Of course not. If e.g. all-integer lists ran 100x faster, you might see an > IntList.py added to the core, and maybe you'd be required to use static > typing *if* you wanted to use that module. There are no consequence-free > changes. It sounds like static typing might become less and less optional as time goes on :-) > I don't much care about Python's speed myself, and have been known to let a > Python program run non-stop for weeks. It is the mother of all bitches, > though, when at the end of the month it dies with an AttributeError because > I typed: > > x.printReslut() > > instead of > > x.printResult() > > at the end. So I try not to do that . To catch these kinds of problems is it really necessary to have a static typing system or could kjpylint be enhanced so that it could catch these kinds of things? I realize that it probably couldn't catch every possible strange case that might arise using Python, but what if it could notice 80% of them and on the rest say, "There's something strange going on around line 562, you might want to double check that you aren't making a mistake there." ? -- Justus Pendleton From fredrik@pythonware.com Sun Nov 29 21:27:12 1998 From: fredrik@pythonware.com (Fredrik Lundh) Date: Sun, 29 Nov 1998 22:27:12 +0100 Subject: [Types-sig] Why I don't like static types in Python Message-ID: <05c601be1bdf$0d4c8c50$f29b12c2@pythonware.com> >Hmmm...the page I read said > >"This list has been created for discussions on issues related to Python's >type system." Sure, and then it outlines the three main subjects for this working group: -- interface definitions (which according to the text on that page would allow programmers to "test object behaviour", provide "a basis for analysis of type safety", and improve the "organization to library and extension classes". ok?). (note: "implicit" interfaces are already used in this fashion in the Python universe -- you've probably seen terms like "sequence", "file iterators", "callables" etc. This attempts to formalize this, so you can at least use "assert" to verify that your function has been given a valid argument...) -- classes vs. types dichotomy (this aims to remove what is essentially a CPython implementation quirk. JPython already does this right. still with me?) -- optional static typing (more on this below). >As far as I can tell this is a working group for "discussions on issues >related to Python's type system" and I think that before anyone starts >discussing how to implement static types it might be good if it were clear >what the goal of such a system might be. I couldn't find a clear description >of that goal in anything that I read. Wrt "static typing", the page I pointed you to still says: "In order to better support a wider range of solution domains, [1] in support of building systems in-the-large with Python [2], and to improve performance [3], there is some interest in developing an optional static typing system for Python. [4]" I didn't write that page, but here's my interpretation of those four points: 1) make it feasible to use Python for more software engineering tasks 2) make it easier to use Python in large, multiprogrammer projects, by adding run-time support for type/interface testing 3) do I have to explain this? alright, here's an illustration: C:\>erase histogram.* C:\>copy src\histogram.py . C:\>python testhisto.py 16.6440000534 seconds C:\>erase histogram.* C:\>spy2c -mq src\histogram.py SPY2C (c) Secret Labs AB 1997-98 histogram.dll ok C:\>python testhisto.py 0.0200001001358 seconds C:\>python testhisto2.py 0.0199999809265 seconds (where testhisto.py uses a function in histogram.py, and testhisto2.py uses PIL). 4) people are already working on this, so there's obviously some interest. people also see the possibility to save a lot of time and money by being able to prototype in Python, before adding what- ever optional information you might need to translate the result to C or machine code. > My post was an attempt to clear up what that goal might be. You actually responded to a post which had nothing to do with static typing. Methinks you're still mixing the three issues. >Perhaps some of these "many highly experienced Python developers" could >answer my questions without resorting to the Authority or Social Proof >methods of persuasion? I still claim that you'll find those answers on the page I keep pointing you to (the page was written after the conference). If you don't, please tell me which parts you find unclear. >I don't care how experienced the doctor is, I usual prefer an >explanation of why he needs to conduct a rectal examination.... Well, some of your arguments have been pretty much like "since I don't need to see my doctor right now, I don't think we need to have doctors at all". >> But in a world where all larger programs are dynamically configured, global >> analysis is a relatively worthless concept. > >Out of curiosity (since I don't know) does this mean that run-time and/or >dynamic recompilation are equally fruitless avenues of inquiry? Why does one thing automatically have to exclude another? >> We're not talking 10% speed increase. We're talking near-C speed. >> And from another perspective, much lower development costs for >> equivalent code. > >I admit that I am skeptical of claims of near-C speed :-) See above. >Java is statically typed and isn't really "near-C speed". It is, if you compare apples with apples -- that is, take a typical Python C extension and rewrite it in Java, and then compile it with a Java2C compiler. (The really interesting issue here isn't replacing applications written entirely in portable assembler with Python, but to replace applications written in Python+C with applications written entirely in Python). >Python claim to be a higher level language than Java so I >have difficulty believing it will succeed where Java fails... >especially since Python's static typing would be voluntary >which makes me believe that it wouldn't be as effective >as Java's mandatory static typing. You mean, like "hey, we don't need doctors since they cannot cure people who don't bother to visit them" ? (I repeat: if you don't like static typing, why not ignore the whole thing? despite all talk about Guido's time machine, nobody's going to use that to add mandatory static typing to the Python interpreter you're using today). >Maybe if someone had a working example of a statically typed, >dynamic, higher level, object oriented language that has near-C >speed I would be more easily convinced..? Why does it have to be everything at once to be useful? I'm playing with statically typed, statically compiled, medium level, procedural Python, which actually happens to be exactly as fast as corresponding C code in most cases. >I have a question (since I don't know), if the static typing is voluntary >then does that mean that nothing in the core python library could use >static typing. If we're talking static typing + static compilation, the entire Python interpreter could be written in Python... >> >Roger Masse writes that static typing provides an 'improved level of type >> >safety and program correctness' but I'm not aware of any empirical evidence >> >of this. >> >> How large is your largest Python application? 10k lines? 20k lines? >> 50k lines? How many people worked on your development team? >> How many design iterations have you had this far? > >Hmmm...this doesn't really answer my concern about the lack of empirical >evidence of the usefulness of static typing. Anecdotal evidence simply isn't >enough as far as I'm concerned. I don't have enough academical background to tell the difference between "anecdotical evidence based on studying lots of large real life projects" and "empirical evidence", so I have to pass on this one. >> >Maybe if someone could give me an example of a real world problem >> >they couldn't solve in Python because it lacked these features it would >> >be easier for me to understand the need we are trying to address here. >> >> http://www.python.org/sigs/types-sig/ > >I am subscribed to the list and have read every message that has ever been >posted to the list (all 54 of them :) Not the mailing list, the page itself. It's all there. >Virtually all of them are about interfaces. Virtually all of them are about >implementation. If you could actually point out which one of those 54 >messages contain "an example of a real world problem they couldn't solve in >Python because it lacked these features it would be easier for me to >understand the need we are trying to address here." If you're still talking static typing, the answer is "anything for which performance is critical". If you need examples, just look at the most popular C extensions, and why people keep using them... Cheers /F fredrik@pythonware.com http://www.pythonware.com From fredrik@pythonware.com Sun Nov 29 21:32:43 1998 From: fredrik@pythonware.com (Fredrik Lundh) Date: Sun, 29 Nov 1998 22:32:43 +0100 Subject: [Types-sig] Why I don't like static types in Python Message-ID: <061701be1bdf$eb52ed50$f29b12c2@pythonware.com> >Hmmm...I'm not sure what the difference between looking up the parameters in >a manual versus looking up the parameters in a prototype are. Seems like >they are exactly the same thing except that the manual is far more likely to >actually convey some useful information... if you'd been a compiler, you would have known the difference... Cheers /F fredrik@pythonware.com http://www.pythonware.com From da@skivs.ski.org Sun Nov 29 22:25:18 1998 From: da@skivs.ski.org (David Ascher) Date: Sun, 29 Nov 1998 14:25:18 -0800 (Pacific Standard Time) Subject: [Types-sig] Why I don't like static types in Python In-Reply-To: <05c601be1bdf$0d4c8c50$f29b12c2@pythonware.com> Message-ID: On Sun, 29 Nov 1998, Fredrik Lundh wrote: > -- classes vs. types dichotomy > > (this aims to remove what is essentially a CPython implementation > quirk. JPython already does this right. Just FYI, that last bit is not quite true. It does go further in the right direction, though. --david From tim_one@email.msn.com Sun Nov 29 22:42:44 1998 From: tim_one@email.msn.com (Tim Peters) Date: Sun, 29 Nov 1998 17:42:44 -0500 Subject: [Types-sig] Why I don't like static types in Python In-Reply-To: <19981129114634.B354@ryoohki> Message-ID: <000101be1be9$94491520$fa9e2299@tim> [Tim] > ... > If e.g. all-integer lists ran 100x faster, you might see an IntList.py > added to the core, and maybe you'd be required to use static typing > *if* you wanted to use that module. There are no consequence-free > changes. [Justus Pendleton] > It sounds like static typing might become less and less optional > as time goes on :-) Oh yes -- but only if it proves to be *so* beneficial that people increasingly clamor for the benefits. So it goes. The flip side is that if it's as useless in practice as you expect it to be, people will ignore it and nothing will change. >> [ x.printReslut() >> vs >> x.printResult() >> ] > To catch these kinds of problems is it really necessary to have a > static typing system No. A declaration scheme may be sufficient to solve it, though, for those motivated enough to use it. There is some number N > 1 of long-standing perceived problems/irritations that optional declarations *may* address adequately and in one stroke. It's unlikely that declarations are the *only* approach to any one of those; it may be the only approach that takes a bite out of all N, though. > or could kjpylint be enhanced so that it could catch these > kinds of things? There are at least 3 flavors of pylint out there that can catch that one already, but only in the "hmm -- this looks fishy" sense of a warning, and also only if you remember to run it and take the time to separate the false alarms from the true by eye -- which, being manual, is also prone to error. > I realize that it probably couldn't catch every possible > strange case that might arise using Python, but what if it could > notice 80% of them and on the rest say, > > "There's something strange going on around line 562, you might > want to double check that you aren't making a mistake there." > > ? That can be done today. If people were happy enough with it, they wouldn't be agitating for more. I think it's widely believed, and with good reason, that the best thing that ever happened to the old "lint" program for K&R C is better compilers and ANSI C changes (incl. prototypes) rendering it useless. languages-grow-to-encompass-the-useful-metatools-they-spawn-ly y'rs - tim From tim_one@email.msn.com Sun Nov 29 22:42:53 1998 From: tim_one@email.msn.com (Tim Peters) Date: Sun, 29 Nov 1998 17:42:53 -0500 Subject: [Types-sig] Type/class unification In-Reply-To: <199811261807.TAA29464@pukapuka.inrialpes.fr> Message-ID: <000401be1be9$998df5a0$fa9e2299@tim> [Vladimir Marangozov] > ... > It's quite unclear to me, even after the 50 types-sig messages > that landed in my inbox, what an (optional) Python interface is, > and why should I feel happier if the Scarecrow (or another) > formula is implemented in some way, Well, I expect just about everyone feels that way -- except for JimF, who seems to know exactly what he wants from interfaces and hasn't budged the width of a mosquito's eyelash since this started. > given the actual implementation of Python's classes. Obviously, > there's some need out there that I don't really understand and there > are limitations in Python that prevent solving the problem easily. I expect we'll understand Jim's view much better when he releases his implementation. Having something concrete to poke at should be a huge clarifier regardless. > ... > An aside: An interesting pointer appeared in c.l.py about a language > called Ruby which has some nice properties. IMHO it worths a look at the > implementation. (Here's a comment from a file "object.c" that bootstraps > the Ruby Universe: > > /* > * Ruby's Class Hierarchy Chart > ... OTOH, from the Ruby reference manual (Class.html): To tell the truth, each class have the unnamed class (which is the meta-class of the class), the class Class is the class of these meta-classes. It is complicated, but it is not important for using Ruby. *That's* sure encouraging . wake-me-up-in-two-years-ly y'rs - tim From tim_one@email.msn.com Sun Nov 29 23:12:31 1998 From: tim_one@email.msn.com (Tim Peters) Date: Sun, 29 Nov 1998 18:12:31 -0500 Subject: [Types-sig] Why I don't like static types in Python In-Reply-To: <4D0A23B3F74DD111ACCD00805F31D8100DB90740@RED-MSG-50> Message-ID: <000701be1bed$bdaec820$fa9e2299@tim> [Tim] > Ahem -- functional languages deliberately constrain their > type systems to make type inference tractable. Python doesn't. [Bill Tutt] > Well, Self had a type infrencing engine written for it, and its > argubably as dynamic, or more dynamic than Python. I'm in a foul mood today. This is comparing apples to dog saliva. Functional languages are deliberately designed to make type inference reasonably efficient and straightforward. Coming up with a type inference system for a language that isn't so designed remains a PhD-level research task: > ... > Ole Agesen's Ph.D. thesis > (http://www.sunlabs.com/technical-reports/1996/abstract-52.html) > "Concrete Type Inferencing: Delivering Object-Oriented Applications". Which, among other things, contains a good discussion of the Hindley-Milner flavor of inferencing, and why that's not good enough for Self (the connection to the above is that functional languages are designed to play nice with Hindley-Milner specifically: design the language to fit the slick algorithms). It took Ole Agesen 187 pages to sketch what he did for Self. That's about 182 pages more than anyone is likely to have time to implement for Python <0.9 wink>. > However several factors make type infrencing Python fairly difficult: > Type infrencing is best used in a closed environment. (akin to > using freeze) > The type infrencing system also benefits greatly from having > detailed type signatures of C extensions. For another take on the subject, see Ken Walker's "The Implementation of an Optimizing Compiler for Icon" (Doctoral Dissertation, The University of Arizona, July 1991). That seems hard to get now, but an extended summary of the type inferencing portion of it can be gotten as the later paper "Type Inference in the Icon Programming Language", by Kenneth Walker and Ralph E. Griswold, from ftp://ftp.cs.arizona.edu/icon/doc/tr93_32.pdf > Which isn't to say it isn't possible, it is, its just complex and time > consuming to implement. > Anybody need a masters thesis topic? ;) It's still PhD-level stuff -- and unlike Self or Icon, Python isn't run by a tenured professor with hordes of doctoral-candidate full-time slave labor. although-jimh-often-does-a-credible-impersonation-ly y'rs - tim From Justus Pendleton Mon Nov 30 02:24:05 1998 From: Justus Pendleton (Justus Pendleton) Date: Sun, 29 Nov 1998 21:24:05 -0500 Subject: [Types-sig] Why I don't like static types in Python In-Reply-To: <000001be1ab6$08270ea0$659e2299@tim>; from Tim Peters on Sat, Nov 28, 1998 at 05:01:13AM -0500 References: <19981127164841.A1370@ryoohki> <000001be1ab6$08270ea0$659e2299@tim> Message-ID: <19981129212405.A564@ryoohki> [Justus Pendleton] > > Hmmm...I'm not sure what the difference between looking up the > > parameters in a manual versus looking up the parameters in a > > prototype are. Seems like they are exactly the same thing > > except that the manual is far more likely to actually convey > > some useful information...like what the hell "int *decpt" :-) [Tim Peters] > The most obvious difference is that a type mismatch between formal arguments > in a prototype and actual arguments in a call can yield a compile-time error > instead of (in C before prototypes) runtime insanity or (in Python) runtime > TypeErrors. [Fredrik Lundh] > if you'd been a compiler, you would have known the difference... As I thought had been clear from my original post, I had tried to break the argument for static typing into its component parts. One of those arguments was "Static types improve readability by making the programmer's intentions more explicit." My comments about documentation being more useful than prototypes were from that viewpoint. I realize that I may not always have been very clear about which part of the argument for static typing I was trying to address :-(, but I think that it could be interesting and possibly informative to deconstruct the argument for static typing and address the arguments one by one. My reason for doing this was to see if perhaps it wasn't the case that there isn't really a need for static typing. -- Justus Pendleton From Justus Pendleton Mon Nov 30 02:31:59 1998 From: Justus Pendleton (Justus Pendleton) Date: Sun, 29 Nov 1998 21:31:59 -0500 Subject: [Types-sig] Why I don't like static types in Python In-Reply-To: <000101be1be9$94491520$fa9e2299@tim>; from Tim Peters on Sun, Nov 29, 1998 at 05:42:44PM -0500 References: <19981129114634.B354@ryoohki> <000101be1be9$94491520$fa9e2299@tim> Message-ID: <19981129213159.B564@ryoohki> On Sun, Nov 29, 1998 at 05:42:44PM -0500, Tim Peters wrote: > [Justus Pendleton] > > It sounds like static typing might become less and less optional > > as time goes on :-) > > Oh yes -- but only if it proves to be *so* beneficial that people > increasingly clamor for the benefits. So it goes. The flip side is that if > it's as useless in practice as you expect it to be, people will ignore it > and nothing will change. Unfortunately (the main reason I really am bothering to argue against static typing at all) I think that this viewpoint misses the fact that most people who start programming in Python have been programming in other languages. They bring the baggage of those languages with them. When DoD Ada programmers start writing C++ code they wonder why C++ can't do bounds checking for arrays. When Visual Basic programmers start writing Java code they wonder why there is no 'Variant' class. Since most of the languages out there (especially the popular ones) use static typing, people will use it in Python regardless of whether they need to. When they learned C they were taught they needed to give a type to every variable....and when they write Python code they will continue to give it a type without thinking of whether it really needs to be there. -- Justus Pendleton From tim_one@email.msn.com Mon Nov 30 04:13:32 1998 From: tim_one@email.msn.com (Tim Peters) Date: Sun, 29 Nov 1998 23:13:32 -0500 Subject: [Types-sig] Why I don't like static types in Python In-Reply-To: <19981129213159.B564@ryoohki> Message-ID: <000601be1c17$caa3a3a0$ed9e2299@tim> [Justus Pendleton] > Unfortunately (the main reason I really am bothering to argue > against static typing at all) I think that this viewpoint misses > the fact that most people who start programming in Python have been > programming in other languages. They bring the baggage of those > languages with them. You may be able to score points where it counts by reminding Guido how well adding a convenient little half-baked lambda turned out <0.9 wink>. > When DoD Ada programmers start writing C++ code they wonder why > C++ can't do bounds checking for arrays. OTOH, pressures like that made Stroustrup add inlined methods to C++ so people could write their own bounds-checking array classes that run at least as fast as Ada's builtin checked arrays. > When Visual Basic programmers start writing Java code they wonder > why there is no 'Variant' class. I figure they wonder more what sick delusion caused Java to claim that the AWT was a usable GUI substrate <0.7 wink>. > Since most of the languages out there (especially the popular ones) > use static typing, people will use it in Python regardless of whether > they need to. When they learned C they were taught they needed to give > a type to every variable....and when they write Python code they will > continue to give it a type without thinking of whether it really needs > to be there. And the harm in that is what? It would be hard to argue it's *bad* that C has declarations, or that it's bad to declare things. What's the worst that can come of it? Some extra typing? The chance that one routine in a dozen could have been used in a more general way if the declarations didn't prevent it? Those are the only two I can think of, and they're not much of a fright. If a casual Python user nevers goes beyond that, I don't mind; and if they're happy too because it lets them do more what they're used to doing, I don't mind that either. If they want to go on from that, any declaration system for Python will most likely include higher-level abstractions like Sequence and Number, at which point the potential polymorphism lost to explicit declarations becomes more academic than real. And if they want to go on from that too, declarations remain optional. Almost everyone who comes to Python writes stuff like for i in range(len(list)): print list[i] at first. After you point out a few times that they can do for thing in list: print thing instead, they eagerly switch styles -- because it's clearer, runs faster, and is less error-prone. Telling them they should stop using declarations because it's more fun to get runtime errors is likely a harder sell . unlike-say-implicit-conversions-declarations-aren't- inherently-evil-ly y'rs - tim From paul@prescod.net Mon Nov 30 05:58:10 1998 From: paul@prescod.net (Paul Prescod) Date: Sun, 29 Nov 1998 23:58:10 -0600 Subject: [Types-sig] Why I don't like static types in Python References: <1299947911-634177@hypernet.com> <000f01be1a75$b80b59c0$6d9e2299@tim> <19981129114634.B354@ryoohki> Message-ID: <366233F2.30A22689@prescod.net> Justus Pendleton wrote: > > To catch these kinds of problems is it really necessary to have a static > typing system or could kjpylint be enhanced so that it could catch these > kinds of things? I realize that it probably couldn't catch every possible > strange case that might arise using Python, but what if it could notice 80% > of them and on the rest say, > > "There's something strange going on around line 562, you might want to double > check that you aren't making a mistake there." The system would not be very usable if every time you ran it it complained about dozens and dozens of lines of perfectly usable Python code. So you would probably need to tell it which objects are mutable and may have arbitrary attributes added to them, which variables can hold multiple types, and so forth. You'll probably want to put these annotations inline for simplicity. When you are done, you have an optional static type system. Paul Prescod - ISOGEN Consulting Engineer speaking for only himself. http://itrc.uwaterloo.ca/~papresco Christmas shopping in a T-Shirt? Toto, I have a feeling we aren't in Canada anymore. From dominic.binks@aethos.co.uk Mon Nov 30 09:23:00 1998 From: dominic.binks@aethos.co.uk (Dominic Binks) Date: Mon, 30 Nov 1998 09:23:00 +0000 Subject: [Types-sig] Re: Types-SIG digest, Vol 1 #13 - 2 msgs References: <199811291700.MAA18538@python.org> Message-ID: <366263F4.31FAD263@aethos.co.uk> Justus Pendleton wrote: > > To catch these kinds of problems is it really necessary to have a static > typing system or could kjpylint be enhanced so that it could catch these > kinds of things? I realize that it probably couldn't catch every possible > strange case that might arise using Python, but what if it could notice 80% > of them and on the rest say, > > "There's something strange going on around line 562, you might want to double > check that you aren't making a mistake there." > Although I think that the aims of a static type system that can cover everything Python can is a great idea. I would just like something just like this. Something that says you passed something that was an int to something that's a string, so on your own head be it. If it had to give up and say eventually with something and declare def f(x,y): to a be a function that takes two anythings (possibly the same, more likely different) and returns an anything, that would be fine. I would say that most code does not follow this example. I would hazard a guess that may be as much as 90% of Python code is regularly type following. Ok, the part that isn't may mean that the rest of the code cannot be typed easily, but I'd still reckon that most functions can be typed quite easily. There are lots of parts of Python that pose problems for type systems. The following example is just one (that I don't think has been given before, but I may be wrong in that) def f(x): if type(x) == StringType: ... elif type(x) == IntType: ... Then x can legitimately take either an Int or a String, which is possibly quite hard to code in a type system. I worked with a language which was a successor to Prolog which had no var predicate. var was used in very similar situations to type(). The decision from the designer was var would never be implemented so you just had to provide two functions rather than one. Obviously this is not possible to do in Python, so the type system would need to be able to handle this. Dominic -- Dominic Binks/Systems Engineer/Aethos Communication Systems Ltd. 400 Park Avenue/Aztec West/Bristol/BS32 4TR/United Kingdom Tel: +44 1454 614455/Fax: +44 1454 620527/Mobile: +44 498 693964 E-mail: dominic.binks@aethos.co.uk From mengx@nielsenmedia.com Mon Nov 30 15:22:30 1998 From: mengx@nielsenmedia.com (Xiangyang Ted Meng) Date: Mon, 30 Nov 1998 10:22:30 -0500 Subject: [Types-sig] optional typing and performance Message-ID: <3662B836.523B@nielsenmedia.com> Hi, I am curious that if it was brought up before and what loop holes it might have. Is it possible to embed optional type declarations in the doc string, something like: def function(a,b,c): ''' (normal documentations) [start of python optional types] function: int a: int, b: String, c: int d: int [end of python optional types] ''' d=a+c return d It might not look good for a language designer. But compared to the proposed ADD-LATER typing ( I only took glimpse of the WEB posting, might have missed something) like below def function: int (a:int,b:String,c:int): ''' (normal documentations) ''' d:int =a+c return d It seems to me that using doc string embedding might be able to offer equivalent solutions to mos of the proposed ones do, but would not require changes of grammar. People not concerned with performance or type checking would never need to know or learn such features since the embedded typing would always be part of value-added documentation(a side benefit) that is not mandatory to read or write, which makes it realy "OPTIONAL". Parser or compiler writers could also feel free to experiment, add or drop these new features or syntax . I strongly hope that the current clean, pleasant yet powerful Python grammar be left as is since these are what attracted me (and likely many others) to use Python (instead of Java ...). Transparent additions of Optional typing and a nice extension framework (and with luck, true threading support) are what could boost Python to be a prime time programming language as well as a platform. I mean major changes should be constrained withing the interpreter/parser/compiler as much as it can be. Thanks -Ted Meng From JOrendorff@ixl.com Mon Nov 30 19:18:21 1998 From: JOrendorff@ixl.com (JOrendorff@ixl.com) Date: Mon, 30 Nov 1998 11:18:21 -0800 Subject: [Types-sig] Re: Types-SIG digest, Vol 1 #13 - 2 msgs References: <199811291700.MAA18538@python.org> <366263F4.31FAD263@aethos.co.uk> Message-ID: <3662EF7D.10220E2D@ixl.com> D. Binks wrote: > There are lots of parts of Python that pose problems for type > systems. [...] > Then x can legitimately take either an Int or a String, which is > possibly quite hard to code in a type system. I think a more compelling example is the variable that can be "either instance of 'foo.Blah' or None". re.match(), for example, returns a value of this type. With no fix, 'x' could just be an "anything", and the hypothetical type system would ignore it. Fix 1. Python could get some sort of mechanism for overloading functions based on argument types, like C++ and Java. (Not sure if this is a good idea or a horrific one.) Fix 2. The type system tries to cover this by allowing you to declare a value of "either type x or type y". Ugh. Fix 3. The particular case "either type x or None" gets special treatment from the type system. 0.5 ugh. I think "no fix" is the right option for now; the proper fix can be applied in 2.1 or later. -- Jason From evan@tokenexchange.com Mon Nov 30 17:36:11 1998 From: evan@tokenexchange.com (Evan Simpson) Date: Mon, 30 Nov 1998 11:36:11 -0600 Subject: [Types-sig] Re: Meta-classes discussion starter Message-ID: <205501be1c87$eca2fda0$2d232cd1@linwin.token.hapenney.com> So is everyone speechless with awe? disgust? ennui? Does this belong elsewhere? nowhere? Throw me a bone, somebody. Honestly, I'd appreciate even a quick line or two saying this won't work / isn't useful / has already been chewed over and dismissed. From gmcm@hypernet.com Mon Nov 30 18:08:47 1998 From: gmcm@hypernet.com (Gordon McMillan) Date: Mon, 30 Nov 1998 13:08:47 -0500 Subject: [Types-sig] Re: Meta-classes discussion starter In-Reply-To: <205501be1c87$eca2fda0$2d232cd1@linwin.token.hapenney.com> Message-ID: <1299690603-14697337@hypernet.com> Evan whines: > > So is everyone speechless with awe? disgust? ennui? > > Does this belong elsewhere? nowhere? > > Throw me a bone, somebody. > 1) I don't think it belongs here, unless you shoe-horn it into the types / classes discussion (or lack thereof). 2) I read it about 6 times, and only sort-of got an idea of where you were headed. 3) Whether or not the intent is good, the syntax is, I think, unmanagable - just way too arcane. 4) The only application of this that was evident was the implementation of "class methods" (in the more esoteric definition of same), which is something only a small number of people seem to miss, (perhaps because only SmallTalkers have any idea what it means). By contrast, the most popular application of metaclasses seems to be to install __getattr__ and __setattr__ hooks, and I don't see how your proposal furthers that. But see (2), above. 5) You also seem to be proposing vast changes in Python's class model. The current metaclass hook is, of course, a wart on the current class model; but one that's backwards compatible. If we're talking about the Python 2 class model, I'm not sure I agree with your definition of metaclasses. Not sure I disagree, either. See (2). if-you-can't-bury-the-family-skeleton-at-least-make-it-dance-ly y'rs - Gordon From M.Faassen@vet.uu.nl Mon Nov 30 18:32:27 1998 From: M.Faassen@vet.uu.nl (Martijn Faassen) Date: Mon, 30 Nov 1998 19:32:27 +0100 Subject: [Types-sig] PRE-PROPOSAL: Static Typing Syntax Message-ID: <3662E4BB.550B8CC1@pop.vet.uu.nl> Hrm, I didn't reply to the Types-SIG but sent it to the previous poster, I think. Here it comes my reply again, for public consumption this time :) I also saw adding 'PROPOSAL' to a message draws attention, so I put that in the title. :) Xiangyang Ted Meng wrote: > > Hi, > > I am curious that if it was brought up before and what loop holes > it might have. Is it possible to embed optional type declarations > in the doc string, something like: > > def function(a,b,c): > ''' > (normal documentations) > > [start of python optional types] > > function: int > a: int, b: String, c: int > d: int > > [end of python optional types] > > ''' > > d=a+c > > return d > > It might not look good for a language designer. > But compared to the proposed ADD-LATER typing ( I only took > glimpse of the WEB posting, might have missed something) > like below > > def function: int (a:int,b:String,c:int): > > ''' > (normal documentations) > ''' > d:int =a+c > > return d I agree the optional static typing proposals I've seen for Python so far look scary; it doesn't look much like Python, as it clutters up stuff a lot. Perhaps it's just that I'm not used to them yet, but I fear it isn't. An alternative to using doc strings is to use a kind of old style K&R C style typing (was this optional too in the beginning? I suppose so!), mixed with some Pascallish, and docstrings: def function(a, b): var: Result: int "The sum of a and b." a, b: int "The things that need to be added." sum: int "Local variable used to temporarily store the sum." """This function uses optional static type checking. """ sum = a + b return sum The 'var' block can safely be ignored and the program should still work. IDEs can even 'fold' it away, and preprocessors can rip all var blocks out. Docstrings can be used to document what variables are used for, so even this would be possible, going beyond just static typing into documentation: def function(): var: Result "Absolutely anything can be returned" I've used 'Result' instead of the function name, for easy copy & pasting of 'var' blocks. I know one shouldn't copy & paste so much it to prevent 'rape & paste programming', but everybody does it at times, right? var blocks can be copied & pasted more easily than intermixed static type annotations. To abstract interface, one could split the 'var' block into an 'signature' (or 'arg') and 'local' block, like this: def function(a, b): signature: a, b: int "Stuff to process" Result: int "Result of the entire calculation." local: foo, bar, baz: int "Lots of helper variables" """This function does a whole lot of silly stuff to a and b. """ foo = a + b * a bar = foo * 12 baz = foo + bar return baz Another function could share the same signature. Python could even figure out which functions are 'fully statically typed', and pass them on to the Python compiler. These functions can still easily be prototyped in Python, and the 'signature' and 'local' blocks would only need to be added when the function stabilizes and we need the speed. A tool could automate this process, by pointing out to the developer which variables in a function haven't been described yet: hocheck test.py Line 173, cannot hyperoptimize function foo; bar has no static type. Line 214, cannot hyperoptimize function xyz; i, j have no static type. Excercises for the reader: * Extend this to classes * Make Python interfaces fit in with this scheme Is-this-more-in-the-spirit-of-Python?-ly yours, Martijn From evan@tokenexchange.com Mon Nov 30 19:06:43 1998 From: evan@tokenexchange.com (Evan Simpson) Date: Mon, 30 Nov 1998 13:06:43 -0600 Subject: [Types-sig] Re: Meta-classes discussion starter Message-ID: <206601be1c94$9400cad0$2d232cd1@linwin.token.hapenney.com> Thanks for the feedback; It appears that I did a really lousy job of expressing myself. Let's see if I can summarize clearly, while answering your points. I did intend my thread to be related to the types/classes discussion. Many of the threads in the archive seem to eventually come around to the idea that if everything is to be an object and have a class (the whole unification thing), then classes must have classes, and to be really tidy and orthogonal, these meta-classes should be well-defined as such (Rather than, say, simply declaring the class of every class to be an empty built-in object MetaClass whose class is itself, and leaving it at that). Given the premise that meta-classes should be to classes as classes currently are to instances, I see two problems. How do you deal with method binding? How do you define methods at different levels (meta-/class/instance) with the same name? These are particularly a problem if you want to initialize your classes and instances with __init__. Meta-classes need a way to spell "__init__ for my classes" and "__init__ for instances of my classes", and "__init__ for my classes" *has* to be a class method if it is to exist. The best solution I could think of was to be able to specify when a method gets bound by giving the number of steps to hold off binding. To avoid name clashes, and to make everything as compatible as possible with current code, keep methods whose binding is deferred tucked away in a pocket attribute. My proposed syntax is ugly, but I couldn't think of an elegant way to spell it. My "best" alternative was stuttering (eg. "def def def __init__(self):"). Bleh. About all you do get out of this scheme is class methods (which I'm accustomed to using in Delphi), something resembling templates, and a bit of extra expressive power. Then again, it doesn't really require much in the way of changes to Python, and can be implemented without breaking any current code (I think). And that's all I had to say :-) -----Original Message----- 1) I don't think it belongs here, unless you shoe-horn it into the types / classes discussion (or lack thereof). 2) I read it about 6 times, and only sort-of got an idea of where you were headed. 3) Whether or not the intent is good, the syntax is, I think, unmanagable - just way too arcane. 4) The only application of this that was evident was the implementation of "class methods" (in the more esoteric definition of same), which is something only a small number of people seem to miss, (perhaps because only SmallTalkers have any idea what it means). By contrast, the most popular application of metaclasses seems to be to install __getattr__ and __setattr__ hooks, and I don't see how your proposal furthers that. But see (2), above. 5) You also seem to be proposing vast changes in Python's class model. The current metaclass hook is, of course, a wart on the current class model; but one that's backwards compatible. If we're talking about the Python 2 class model, I'm not sure I agree with your definition of metaclasses. Not sure I disagree, either. See (2). From ldl@LDL.HealthPartners.COM Mon Nov 30 19:23:15 1998 From: ldl@LDL.HealthPartners.COM (LD Landis) Date: Mon, 30 Nov 1998 13:23:15 -0600 (CST) Subject: [Types-sig] Re: Types-SIG digest, Vol 1 #14 - 12 msgs In-Reply-To: <199811301700.MAA29572@python.org> from "types-sig-admin@python.org" at Nov 30, 98 12:00:49 pm Message-ID: <199811301923.NAA00343@LDL.HealthPartners.COM> Hi, Ted's idea of embedding the type informationin the docstring is interesting, but this statement really rings true for me as well! Please, let's "Keep It Simple Somehow"! Having it in a special place in the docstring, or allowing for some SGML/XML markups to introduce it... Then you could have any number of equivalent, non-intrusive solutions. Xiangyang Ted Meng wrote: > > I strongly hope that the current clean, pleasant yet powerful > Python grammar > be left as is since these are what attracted me (and likely many others) > to use Python (instead of Java ...). Transparent additions of > Optional typing and a nice extension framework (and with luck, true > threading support) are what could boost Python to be a prime time > programming language as well as a platform. I mean major changes > should be constrained withing the interpreter/parser/compiler as much > as it can be. > > Thanks > > -Ted Meng > -- Cheers, --ldl ----------------------------------------------------------------------------- LD Landis ldl@HealthPartners.Com N0YRQ Voice 612/883-5511 Fax 612/883-6363 HealthPartners, 8100 34th Avenue So, PO Box 1309, Minneapolis, MN 55440-1309 Shape your life not from your memories, but from your hopes. (Borrowed) ----------------------------------------------------------------------------- From guido@CNRI.Reston.VA.US Mon Nov 30 19:35:18 1998 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Mon, 30 Nov 1998 14:35:18 -0500 Subject: [Types-sig] Re: Meta-classes discussion starter In-Reply-To: Your message of "Mon, 30 Nov 1998 13:06:43 CST." <206601be1c94$9400cad0$2d232cd1@linwin.token.hapenney.com> References: <206601be1c94$9400cad0$2d232cd1@linwin.token.hapenney.com> Message-ID: <199811301935.OAA03770@eric.cnri.reston.va.us> > I did intend my thread to be related to the types/classes discussion. Many > of the threads in the archive seem to eventually come around to the idea > that if everything is to be an object and have a class (the whole > unification thing), then classes must have classes, and to be really tidy > and orthogonal, these meta-classes should be well-defined as such (Rather > than, say, simply declaring the class of every class to be an empty built-in > object MetaClass whose class is itself, and leaving it at that). > > Given the premise that meta-classes should be to classes as classes > currently are to instances, I see two problems. How do you deal with > method binding? How do you define methods at different levels > (meta-/class/instance) with the same name? These are particularly a problem > if you want to initialize your classes and instances with __init__. > Meta-classes need a way to spell "__init__ for my classes" and "__init__ for > instances of my classes", and "__init__ for my classes" *has* to be a class > method if it is to exist. Have you read my post "ASCII art" in the python-classes@egroups.com mailing list? It's very relevant. I'll repeat it here, because it's such a drag to dig it out of the egroups archives (and their HTML screws up the ASCII art!). But you should still read that group's archives, at http://www.egroups.com/list/python-classes/. ------------------------------ repost ------------------------------ Subject: [python-classes] ASCII art From: Guido van Rossum To: python-classes@egroups.com Date: Tue, 27 Oct 1998 19:08:14 -0500 I've been working on a response for days now, and I'm giving up. Instead, I'll try to draw some stuff that I drew on a whiteboard today in front of a bunch of Norwegian and local Pythoneers. Consider instance x of class C, derived from class B. This is the ideal diagram: +-----------+ | x | +-----------+ | | v +-----------+ +-----------+ | C |---->| B | +-----------+ +-----------+ \ / \ / v v +-----------+ | class | +-----------+ I use the following convention: an arrow pointing down goes from an object to its meta-object, class or type. An arrow pointing right goes from a class to its base class. The idea is that you follow the down arrow to get to an object's behavior -- so x's behavior comes from C, and C's behavior comes from 'class' (and so does B's). You follow the arrow right to delegate attributes to a base class. (With multiple inheritance, you draw multipl arrows.) Note that the delegation from C to B will allow x to find behavior in B if it isn't found in C. In the current CPython implementation, things look a bit different: +-----------+ | x | +-----------+ / | / | / v / +-----------+ +-----------+ / | C |---->| B | / +-----------+ +-----------+ v \ / *===========* \ / I instance I v v *===========* *===========* \ I class I \ *===========* \ / \ / v v *===========* I type I *===========* That is, x has *two* behavior-defining links: one to the type 'instance', one to the class C. (I've drawn type objects in a different style here.) This is what causes a lot of grief. I'd like to move to the first diagram. It can be extended downwards and to the right. You could have as many levels of metaclasses as you want, and it would all be well defined: behavior comes from below, delegated behavior from the right. I've got to go soon, and I want this to go out tonight, so I'll just add some quick notes. - I like David's idea of specifying the metaclass in the class statement. Specifying the base class is specifying the arrow to the right; specifying the metaclass is specifying the arrow down. - There is a difference between class methods (which specify behavior of the class, and are defined in the metaclass) and C++/Java style static methods (which should be defined in the class using a special syntax so that they don't belong to a specific instance; however, as in C++/Java, it should still be *possible* to invoke them via the instance, as well as via the class.) [I had a longer rant about this, but no time to condense it now.] - There could be a class "Object" which is used as the default right arrow. - There could also be a class "Turtles-all-the-way-down" which is used as the default arrow down. - I don't know if Object and Turtles are the same object. I know that each has a right arrow to Object and each has a down arrow to Turtles; they *could* be related like this: ------- --------- ------------------ | | | \ | | | v v \ | | | +---------+ \ \->+--------+ | | | Turtles |------------->| Object |---/ | +---------+ \ +--------+ | | \ | \ / \ / ---------- -------- (Heh! :-) - There are plenty of problems left. Later, after my drumming class :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From evan@tokenexchange.com Mon Nov 30 20:05:09 1998 From: evan@tokenexchange.com (Evan Simpson) Date: Mon, 30 Nov 1998 14:05:09 -0600 Subject: [Types-sig] Re: Meta-classes discussion starter Message-ID: <208801be1c9c$bbdfe100$2d232cd1@linwin.token.hapenney.com> Hmm. I guess I need to clarify my clarification :-) >Have you read my post "ASCII art" in the python-classes@egroups.com >mailing list? It's very relevant. I'll repeat it here, because it's >such a drag to dig it out of the egroups archives (and their HTML >screws up the ASCII art!). But you should still read that group's >archives, at http://www.egroups.com/list/python-classes/. When I said "Many of the threads in the archive...", I was in fact referring to the python-classes egroups archive. Your ASCII art post was my mental jumping-off point for all of my thoughts on meta-classes. That said, I couldn't make any sense of either of the following: >- I like David's idea of specifying the metaclass in the class >statement. Specifying the base class is specifying the arrow to the >right; specifying the metaclass is specifying the arrow down. Perhaps I'm thinking too statically, but to my mind, you don't specify the metaclass of a class any more than you specify the class of an instance (by "calling" the (meta-)class). This is essentially where my two meta-class problems arise. >- There is a difference between class methods (which specify behavior >of the class, and are defined in the metaclass) and C++/Java style >static methods (which should be defined in the class using a special >syntax so that they don't belong to a specific instance; however, as >in C++/Java, it should still be *possible* to invoke them via the >instance, as well as via the class.) [I had a longer rant about this, >but no time to condense it now.] Where would this difference lie, other than in the point of definition? Or are you simply dividing methods which bind to classes from those which bind to instances? >- There are plenty of problems left. Later, after my drumming class :-) I never saw any deeper exploration of these ideas. If there was, could someone point me at it? From evan@tokenexchange.com Mon Nov 30 20:20:24 1998 From: evan@tokenexchange.com (Evan Simpson) Date: Mon, 30 Nov 1998 14:20:24 -0600 Subject: [Types-sig] Re: Meta-classes discussion starter Message-ID: <209201be1c9e$dd6755e0$2d232cd1@linwin.token.hapenney.com> Ah. Re-reading what I just wrote, I think I've found where I went astray. Given the chain from meta-class to instance MMC -> MC -> C -> c, I have been thinking of the intermediate classes MC and C as being implicit in the definition of MMC, rather than as independently defined classes. I can see how defining them, and specifiying their meta-classes, would solve the binding and naming problems. You could even have multiple meta-classes just as you have multiple inheritance. All you need now is a way to spell static methods. Thanks for helping me work that out. From gmcm@hypernet.com Mon Nov 30 21:34:16 1998 From: gmcm@hypernet.com (Gordon McMillan) Date: Mon, 30 Nov 1998 16:34:16 -0500 Subject: [Types-sig] Re: Meta-classes discussion starter In-Reply-To: <209201be1c9e$dd6755e0$2d232cd1@linwin.token.hapenney.com> Message-ID: <1299678281-15438179@hypernet.com> Evan Simpson, after re-reading Guido's ASCII art post, writes: > I can see how defining them, and specifiying their meta-classes, > would solve the binding and naming problems. You could even have > multiple meta-classes just as you have multiple inheritance. All > you need now is a way to spell static methods. > > Thanks for helping me work that out. Don't you just hate stretching your brain to the limits, and then finding out the Guido has zipped right past you? Much as I hate to admit it, I think Guido's ideas solve a number of problems with thinking about metaclasses, and opens the door to "Meta Object Protocol" type behavior, which is, I think, what we're really after. One possibe spelling of this (which I don't have much hope for, but at least it's better than David's ): class MyClass(bases=(MyBase,), turtles=(Trace,)): blah... Now my "what"s come from MyClass and MyBase, but my "how"s come from Trace. Voila, (relatively) painless __getattr__ / __setattr__ hooks (courtesy of turtles). Now I can see where SmallTalk-ish "class methods" fit (in turtles), but not where (or if) those silly "static" type class methods fit. Not that I have much attachment to them, but they seem to be a commonly desired feature. Best of all, you don't have to use that damn m-word, or wonder how an object can be both an instance and a class at the same time. ride-a-turtle-and-take-the-strain-off-your-brain-ly y'rs - Gordon From just@letterror.com Mon Nov 30 22:08:16 1998 From: just@letterror.com (Just van Rossum) Date: Mon, 30 Nov 1998 23:08:16 +0100 Subject: [Types-sig] Re: Meta-classes discussion starter In-Reply-To: <208801be1c9c$bbdfe100$2d232cd1@linwin.token.hapenney.com> Message-ID: At 2:05 PM -0600 11/30/98, Evan Simpson wrote: >Perhaps I'm thinking too statically, but to my mind, you don't specify the >metaclass of a class any more than you specify the class of an instance (by >"calling" the (meta-)class). This is essentially where my two meta-class >problems arise. Hmmm... - when you call a class it returns a new instance - when you call a metaclass it returns a new class That could make for some interesting Python 2.0 syntax: class MyClass = MyMetaClass(BaseClass1, ..., BaseClassN): ...etc. where class MyClass(BaseClass1, ..., BaseClassN): ... would be a shortcut for class MyClass = ClassClass(BaseClass1, ..., BaseClassN): ...etc. (where ClassClass is the default metaclass) (Sorry, I didn't read the egroups archive so I have not seen David's syntax proposal.) Not sure whether I like that... On a related note, while studying Guido's MetaClass essay and the accompanying code (initially my head exploded every five minutes, today I got by with only once) an idea for Yet Another MetaHook occurred to me. After I sortof understood Guido's stuff I became annoyed by the fact that the inheritance syntax gets abused to specify a metaclass. Which has as a side effect that you _have_ to make an instance of the metaclass to be able to use it as a metaclass, and that you _have_ to use the original class for real subclassing. (And in turn you have to instantiate that subclass to use it.) Two objects to refer to the same concept: very confusing (at least for me ;-). While digging in ceval.c and understanding how the Don Beaudry and Guido's own hook actually work I came up with mine, which I define thusly: If the __dict__ of the class that is being built has a __class__ key, call that thing with (name, bases, methods) Which means you specify a metaclass like this: class MyClass: __class__ = MyMetaClass Not exactly pretty, but at least it doesn't look like inheritance. It also means that if you want to subclass from MyMetaClass you write class MyOtherMetaClass(MyMetaClass): ... With Guido's hook the above looks like this: MyMeta = MyMetaClass("MyMeta", (), {}) class MyClass(MyMeta): ... class MyOtherMetaClass(MyMetaClass): ... MyOtherMeta = MyOtherMetaClass("MyOtherMeta", (), {}) The cool thing is that inheritance from a class that has a meta class works like you would expect, because of Guido's hook... This is not theory: I've got it working right here, and I seriously propose it for 1.6, if not 1.5.2. (If anyone wants to play, I could post a patch.) Then... ...I tried to emulate the Python class/instance model using metaclasses, but I got stuck several times due to severe brain damage alternated with head explosion. But I believe now that that's not the fault of metaclasses in Python, but that it is just really hard to do, largely because of problems Evan noticed, too. Simple example, if you do repr(object) the current Python semantics say: get the __repr__ attribute of object. Which means if object is an instance, first it's __dict__ will be searched, then its __class__ and then it's base classes. Usually, object.__repr__ is the same as object.__class__.__repr__. Now what if you want to repr() the class? The easiest thing I could think of to define __repr__ (and some others, like __call__) as special, and _never_ search the object itself, but only its class. So object.__repr__ lives in object.__class__ and object.__class__.__repr__ lives in object.__class__.__class__.__repr__. Phew. Anyway, thanks for listening... Just