From paulp@ActiveState.com Sun Mar 11 23:23:07 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Sun, 11 Mar 2001 15:23:07 -0800 Subject: [Types-sig] Revive the types sig? Message-ID: <3AAC08DB.9D4E96B4@ActiveState.com> I have been involved with the types-sig for a long time and it has consumed countless hours out of the lives of many brilliant people. I strongly believe that it will only ever work if we change some of fundamental assumptions, goals and procedures. At next year's conference, I do not want to be at the same place in the discussion that we were this year, and last year, and the year before. The last time I thought I could make progress through sheer effort. All that did was burn me out and stress out my wife. We've got to work smarter, not harder. The first thing we need to adjust is our terminology and goals. I think that we should design a *parameter type annotation* system that will lead directly to better error checking *at runtime*, better documentation, better development environments an so forth. Checking types *at compile time* should be considered a tools issue that can be solved by separate tools. I'm not going to say that Python will NEVER have a static type checking system but I would say that that shouldn't be a primary goal. I've reversed my opinion on this issue. Hey, even Guido makes mistakes. I think that if the types-sig is going to come up with something useful this time, we must observe a few principles that have proven useful in developing Python: 1. Incremental development is okay. You do not need the end-goal in mind before you begin work. Python today is very different than it was when it was first developed (not as radically different than some languages, but still different). 2. It is not necessary to get everything right. Python has some warts. Some are easier to remove than others but they can all be removed eventually. We have to get a type system done, test it out, and then maybe we have to remove the warts. We may not design a perfect gem from the start. Perfection is a goal, not a requirement. 3. Whatever feature you believe is absolutely necessary to a decent type system probably is not. There are no right or wrong answers, only features that work better or worse than other features. It is important to understand that a dynamically-checked type annotation system is just a replacement for assertions. Anything that cannot be expressed in the type system CAN be expressed through assertions. For instance one person might claim that the type system needs to differentiate between 32 bit integers and 64 bit integers. But if we do not allow that differentiation directly in the type system, they could do that in assertions. C'est la vie. This is not unique to Python. Languages like C++ and Java also have type test and type assertion operators to "work around" the limitations of their type systems. If people who have spent their entire lives inventing static type checking systems cannot come up with systems that are 100% "complete" then we in the Python world should not even try. There is nothing wrong with using assertions for advanced type checks. For instance, if you try to come up with a type system that can define the type of "map" you will probably come up with something so complicated that it will never be agreed upon or implemented. (Python's map is much harder to type-declare than that of functional languages because the function passed in must handle exactly as many arguments as the unbounded number of sequences that are passed as arguments to map.) Even if we took an extreme position and ONLY allowed type annotations for basic types like strings, numbers and sequences, Python would still be a better language. There are thousands of instances of these types in the standard library. If we can improve the error checking and documentation of these methods we have improved on the status quo. Adding type annotations for the other parameters could wait for another day. ---- In particular there are three features that have always exploded into unending debates in the past. I claim that they should temporarily be set aside while we work out the basics. A) Parameterized types (or templates): Parameterized types always cause the discussion to spin out of control as we discuss levels and types of parameterizability. A type system can be very useful with parameterization. For instance, Python itself is written in C. C has no parameterizability. Yet C is obviously still very useful (and simple!). Java also does not yet have parameterized types and yet it is the most rapidly growing statically typed programming language! It is also important to note that parameterized types are much, much more important in a language that "claims" to catch most or all type errors at compile time. Python will probably never make that claim. If you want to do a more sophisticated type check than Python allows, you should do that in an assertion: assert Btree.containsType(String) Once the basic type system is in place, we can discuss the importance of parameterized types separately later. Once we have attempted to use Python without them, we will understand our needs better. The type system should not prohibit the addition of parameterized types in the future. A person could make a strong argument for allowing parameterization only of basic types ("list of string", "tuple of integers") but I think that we could even postpone this for the future. B) Static type checking: Static type warnings are important and we want to enable the development of tools that will detect type errors before applications are shipped. Nevertheless, we should not attempt to define a static type checking system for Python at this point. That may happen in the future or never. Unlike Java or C++, we should not require the Python interpreter itself to ever reject code that "might be" type incorrect. Other tools such as linters and IDEs should handle these forms of whole-program type-checks. Rather than defining the behavior of these tools in advance, we should leave that as a quality of implementation issue for now. We might decide to add a formally-defined static type checking to Python in the future. Dynamically checked annotations give us a starting point. Once again, I think that the type system should be defined so that annotations could be used as part of a static type checking system in the future, should we decide that we want one. C) Attribute-value and variable declarations: In traditional static type checking systems, it is very important to declare the type for attributes in a class and variables in a function. This feature is useful but it is fairly separable. I believe it should wait because it brings up a bunch of issues such as read-only attributes, cross-boundary assignment checks and so forth. I propose that the first go-round of the types-sig should ONLY address the issue of function signatures. Let's discuss my proposal in the types-sig. Executive summary: * incremental development policy * syntax for parameter type declarations * syntax for return type declarations * optional runtime type checking * goals are better runtime error reporting and method documentation Deferred for future versions (or never): * compile-time type checking * parameterized types * declarations for variables and attributes http://www.python.org/sigs/types-sig/ -- Python: Programming the way Guido indented it. From jeremy@alum.mit.edu Mon Mar 12 01:22:04 2001 From: jeremy@alum.mit.edu (Jeremy Hylton) Date: Sun, 11 Mar 2001 20:22:04 -0500 (EST) Subject: [Types-sig] Re: [Python-Dev] Revive the types sig? In-Reply-To: <3AAC08DB.9D4E96B4@ActiveState.com> References: <3AAC08DB.9D4E96B4@ActiveState.com> Message-ID: <15020.9404.557943.164934@w221.z064000254.bwi-md.dsl.cnc.net> >>>>> "PP" == Paul Prescod writes: PP> Let's discuss my proposal in the types-sig. Executive summary: PP> * incremental development policy PP> * syntax for parameter type declarations PP> * syntax for return type declarations PP> * optional runtime type checking PP> * goals are better runtime error reporting and method PP> documentation If your goal is really the last one, then I don't see why we need the first four <0.9 wink>. Let's take this to the doc-sig. I have never felt that Python's runtime error reporting is all that bad. Can you provide some more motivation for this concern? Do you have any examples of obscure errors that will be made clearer via type declarations? The best example I can think of for bad runtime error reporting is a function that expects a sequence (perhaps of strings) and is passed a string. Since a string is a sequence, the argument is treated as a sequence of length-1 strings. I'm not sure how type declarations help, because: (1) You would usually want to say that the function accepts a sequence -- and that doesn't get you any farther. (2) You would often want to say that the type of the elements of the sequence doesn't matter -- like len -- or that the type of the elements matters but the function is polymorphic -- like min. In either case, you seem to be ruling out types for these very common sorts of functions. If documentation is really the problem you want to solve, I imagine we'd make much more progress if we could agree on a javadoc-style format for documentation. The ability to add return-type declarations to functions and methods doesn't seem like much of a win. Jeremy From michel@digicool.com Mon Mar 12 02:05:48 2001 From: michel@digicool.com (Michel Pelletier) Date: Sun, 11 Mar 2001 18:05:48 -0800 (PST) Subject: [Types-sig] Re: [Python-Dev] Revive the types sig? In-Reply-To: <3AAC08DB.9D4E96B4@ActiveState.com> Message-ID: On Sun, 11 Mar 2001, Paul Prescod wrote: > Let's discuss my proposal in the types-sig. Executive summary: > > * incremental development policy > * syntax for parameter type declarations > * syntax for return type declarations > * optional runtime type checking > * goals are better runtime error reporting and method documentation I could be way over my head here, but I'll try to give you my ideas. I've read the past proposals for type declarations and their syntax, and I've also read a good bit of the types-sig archive. I feel that there is not as much benefit to extending type declarations into the language as their is to interfaces. I feel this way because I'm not sure what benefit this has over an object that describes the types you are expecting and is associated with your object (like an interface). The upshot of having an interface describe your expected parameter and return types is that the type checking can be made as compile/run-time, optional/madatory as you want without changing the language or your implementation at all. "Strong" checking could be done during testing, and no checking at all during production, and any level in between. A disadvantage of an interface is that it is a seperate, additional step over just writing code (as are any type assertions in the language, but those are "easier" inline with the implementation). But this disadvantage is also an upshot when you immagine that the interface could be developed later, and bolted onto the implementation later without changing the implementation. Also, type checking in general is good, but what about preconditions (this parameter must be an int > 5 < 10) and postconditions and other conditions one does now with assertions. Would these be more language extensions in your propsal? As I see it, interfaces satify your first point, remove the need for your second and third point, satify your fourth point, and meet the goals of your fifth. Nice to meet you at the conference, -Michel From ping@lfw.org Mon Mar 12 05:18:06 2001 From: ping@lfw.org (Ka-Ping Yee) Date: Sun, 11 Mar 2001 21:18:06 -0800 (PST) Subject: [Types-sig] Re: [Python-Dev] Revive the types sig? In-Reply-To: Message-ID: On Sun, 11 Mar 2001, Michel Pelletier wrote: > As I see it, interfaces satify your first point, remove the need for your > second and third point, satify your fourth point, and meet the goals of > your fifth. For the record, here is a little idea i came up with on the last day of the conference: Suppose there is a built-in class called "Interface" with the special property that whenever any immediate descendant of Interface is sub-classed, we check to make sure all of its methods are overridden. If any methods are not overridden, something like InterfaceException is raised. This would be sufficient to provide very simple interfaces, at least in terms of what methods are part of an interface (it wouldn't do any type checking, but it could go a step further and check the number of arguments on each method). Example: >>> class Spam(Interface): ... def islovely(self): pass ... >>> Spam() TypeError: interfaces cannot be instantiated >>> class Eggs(Spam): ... def scramble(self): pass ... InterfaceError: class Eggs does not implement interface Spam >>> class LovelySpam(Spam): ... def islovely(self): return 1 ... >>> LovelySpam() Essentially this would replace the convention of writing a whole bunch of methods that raise NotImplementedError as a way of describing an abstract interface, making it a bit easier to write and causing interfaces to be checked earlier (upon subclassing, rather than upon method call). It should be possible to implement this in Python using metaclasses. -- ?!ng "Computers are useless. They can only give you answers." -- Pablo Picasso From paulp@ActiveState.com Mon Mar 12 06:27:16 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Sun, 11 Mar 2001 22:27:16 -0800 Subject: [Types-sig] Re: Revive the types sig? References: <3AAC08DB.9D4E96B4@ActiveState.com> <15020.9404.557943.164934@w221.z064000254.bwi-md.dsl.cnc.net> Message-ID: <3AAC6C44.D63BE517@ActiveState.com> (followed-up to types-sig) Jeremy Hylton wrote: > > >>>>> "PP" == Paul Prescod writes: > > PP> Let's discuss my proposal in the types-sig. Executive summary: > > PP> * incremental development policy > PP> * syntax for parameter type declarations > PP> * syntax for return type declarations > PP> * optional runtime type checking > PP> * goals are better runtime error reporting and method > PP> documentation > > If your goal is really the last one, then I don't see why we need the > first four <0.9 wink>. Let's take this to the doc-sig. > > I have never felt that Python's runtime error reporting is all that > bad. Can you provide some more motivation for this concern? Do you > have any examples of obscure errors that will be made clearer via type > declarations? I've seen hundreds of examples but here's one that I produced in 30 seconds in the interpreter: >>> a = None >>> # 10,000 lines of code ... >>> urllib.urlopen(a) Traceback (most recent call last): File "", line 1, in ? File "c:\python20\lib\urllib.py", line 61, in urlopen return _urlopener.open(url) File "c:\python20\lib\urllib.py", line 139, in open fullurl = unwrap(fullurl) File "c:\python20\lib\urllib.py", line 857, in unwrap url = string.strip(url) File "c:\python20\lib\string.py", line 80, in strip return s.strip() AttributeError: 'None' object has no attribute 'strip' >>> The interpreter should have caught the problem in the urlopen, not several calls down. Also, it would be nice if a type checker or IDE could detect a potential problem by inspecting the assignments to "a" and the declaration of urlopen. Of course Python is a dynamic language so you can't always depend on that but sometimes it would work. > ... > If documentation is really the problem you want to solve, I imagine > we'd make much more progress if we could agree on a javadoc-style > format for documentation. The ability to add return-type declarations > to functions and methods doesn't seem like much of a win. The first two priorities are err and doc. There are two other lesser goals: "opt" (through type inference) and "idl" which is about integrating Python with strongly-type systems such as SOAP, .NET, COM, CORBA and all of the statically typed programming languages. e.g. for Jython it would be great if you could generate strongly typed Java classes if you wanted to. -- Python: Programming the way Guido indented it. - (originated with Skip Montanaro?) From paulp@ActiveState.com Mon Mar 12 06:35:28 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Sun, 11 Mar 2001 22:35:28 -0800 Subject: [Types-sig] Re: [Python-Dev] Revive the types sig? References: Message-ID: <3AAC6E30.CFB80A63@ActiveState.com> Michel Pelletier wrote: > > I could be way over my head here, but I'll try to give you my ideas. You're not over your head and I'd like to invite you to the types-sig. > I've read the past proposals for type declarations and their > syntax, and I've also read a good bit of the types-sig archive. > > I feel that there is not as much benefit to extending type declarations > into the language as their is to interfaces. I feel this way because I'm > not sure what benefit this has over an object that describes the types you > are expecting and is associated with your object (like an interface). I strongly believe that interfaces are a Good Thing. If you DC guys weren't working on them I'd spend my own time working on them. But they in no way solve the problems of the rest of the types-sig. There are many open problems: * what is the syntax for describing parameters to an interface? * what is the type hierarchy (or dag) underlying that syntax? * how do we unify the type, class and interface worlds? * how do we specify the parameters of functions that are NOT on classes. * how do we specifry the parameters of classes that do NOT have associated interfaces? > ... > A disadvantage of an interface is that it is a seperate, additional step > over just writing code (as are any type assertions in the language, but > those are "easier" inline with the implementation). But this > disadvantage is also an upshot when you immagine that the interface could > be developed later, and bolted onto the implementation later without > changing the implementation. Agreed. There are good reasons for wanting "inline" declarations (ease of programming and prototyping) and good reasons for wanting separate interfaces (especially for reuse). I think that this is a strong argument that we should support both. > Also, type checking in general is good, but what about preconditions (this > parameter must be an int > 5 < 10) and postconditions and other conditions > one does now with assertions. Would these be more language extensions in > your propsal? One of my governing philosophies about type systems is that you can always make them more complex/sophisticated. I don't see that pre and post-conditions are so important that they need first-class syntax. Assertions are sufficient. > As I see it, interfaces satify your first point, remove the need for your > second and third point, satify your fourth point, and meet the goals of > your fifth. To recap: > * incremental development policy Fine. > * syntax for parameter type declarations How do we do parameter type declarations in scarecrow? > * syntax for return type declarations How do we do return type declarations in scarecrow? > Nice to meet you at the conference, > > -Michel Likewise! -- Python: Programming the way Guido indented it. - (originated with Skip Montanaro?) From uche.ogbuji@fourthought.com Mon Mar 12 07:11:27 2001 From: uche.ogbuji@fourthought.com (Uche Ogbuji) Date: Mon, 12 Mar 2001 00:11:27 -0700 Subject: [Types-sig] Re: [Python-Dev] Revive the types sig? In-Reply-To: Message from Jeremy Hylton of "Sun, 11 Mar 2001 20:22:04 EST." <15020.9404.557943.164934@w221.z064000254.bwi-md.dsl.cnc.net> Message-ID: <200103120711.AAA09711@localhost.localdomain> Jeremy Hylton: > If documentation is really the problem you want to solve, I imagine > we'd make much more progress if we could agree on a javadoc-style > format for documentation. The ability to add return-type declarations > to functions and methods doesn't seem like much of a win. I know this isn't the types SIG and all, but since it has come up here, I'd like to (once again) express my violent disagreement with the efforts to add static typing to Python. After this, I won't pursue the thread further here. I used to agree with John Max Skaller that if any such beast were needed, it should be a more general system for asserting correctness, but I now realize that even that avenue might lead to madness. Python provides more than enough power for any programmer to impose their own correctness tests, including those for type-safety. Paul has pointed out to me that the goal of the types SIG is some mechanism that would not affect those of us who want nothing to do with static typing; but my fear is that once the decision is made to come up with something, such considerations might be the first out the window. Indeed, the last round of talks produced some very outre proposals. Type errors are not even close to the majority of those I make while programming in Python, and I'm quite certain that the code I've written in Python is much less buggy than code I've written in strongly-typed languages. Expressiveness, IMO, is a far better aid to correctness than artificial restrictions (see Java for the example of school-marm programming gone amok). If I understand Jeremy correctly, I am in strong agreement that it is at least worth trying the structured documentation approach to signalling pre- and post-conditions before turning Python into a rather different language. -- Uche Ogbuji Principal Consultant uche.ogbuji@fourthought.com +1 303 583 9900 x 101 Fourthought, Inc. http://Fourthought.com 4735 East Walnut St, Ste. C, Boulder, CO 80301-2537, USA Software-engineering, knowledge-management, XML, CORBA, Linux, Python From tim.one@home.com Mon Mar 12 07:30:03 2001 From: tim.one@home.com (Tim Peters) Date: Mon, 12 Mar 2001 02:30:03 -0500 Subject: [Types-sig] RE: Revive the types sig? In-Reply-To: <200103120711.AAA09711@localhost.localdomain> Message-ID: Could we please prune followups on this to the Types-SIG now? I don't really need to see three copies of every msg, and everyone who has the slightest interest in the topic should already be on the Types-SIG. grumpily y'rs - tim From mwh21@cam.ac.uk Mon Mar 12 07:55:07 2001 From: mwh21@cam.ac.uk (Michael Hudson) Date: 12 Mar 2001 07:55:07 +0000 Subject: [Types-sig] Revive the types sig? In-Reply-To: Paul Prescod's message of "Sun, 11 Mar 2001 15:23:07 -0800" References: <3AAC08DB.9D4E96B4@ActiveState.com> Message-ID: Paul Prescod writes: > I think that if the types-sig is going to come up with something > useful this time, we must observe a few principles that have proven > useful in developing Python: Here's mine: don't waste all your energies on discussion here. Spend at least some of them actually coming up with something concrete and for bonus points implement it, or we'll just have 1999-12 again. Then we can shout about that. I'm not sure design-by-commitee can ever work in a vacuum and fairly sure it can't when there is an unbounded number of commitee members. Cheers, M. -- If comp.lang.lisp *is* what vendors are relying on to make or break Lisp sales, that's more likely the problem than is the effect of any one of us on such a flimsy marketing strategy... -- Kent M Pitman, comp.lang.lisp From michel@digicool.com Mon Mar 12 08:23:53 2001 From: michel@digicool.com (Michel Pelletier) Date: Mon, 12 Mar 2001 00:23:53 -0800 (PST) Subject: [Types-sig] Re: [Python-Dev] Revive the types sig? In-Reply-To: <3AAC6E30.CFB80A63@ActiveState.com> Message-ID: Prelude: I've just been digging a bit through the types-sig archives. Aye carumba, we are opening a big-arsed can of worms up here. In a way, we may be leading ourselves down the same path of confusion, maybe what we need here is a very high level description of vision, scope, and goals, perchance a use case or two. At DC, we like to do these things in Wikis so that they don't get lost in the archives. I'd be happy to create a Wiki for this. On Sun, 11 Mar 2001, Paul Prescod wrote: > Michel Pelletier wrote: > > > > I could be way over my head here, but I'll try to give you my ideas. > > You're not over your head and I'd like to invite you to the types-sig. Just subscribed, thanks! > I strongly believe that interfaces are a Good Thing. If you DC guys > weren't working on them I'd spend my own time working on them. But they > in no way solve the problems of the rest of the types-sig. There are > many open problems: > > * what is the syntax for describing parameters to an interface? There isn't one as they ride piggy-back on the syntax of a class. If they had their own syntax, that could change. But the framework *does* support the notion of tagging any interface element with any kind of data, like UML. One of these tagged bits could easily be a parameter description of any kind, like type, pre/postcondition, unit test, example code, links to external resources etc. How this is spelled in syntax is not defined, and for many things (perhaps other than type) it should not have a syntax. > * what is the type hierarchy (or dag) underlying that syntax? Ya lost me there. > * how do we unify the type, class and interface worlds? These seem like seperate issues to me. > * how do we specify the parameters of functions that are NOT on > classes. Good point, interfaces don't do that. But now that I think about it, it seems possible, I've just never thought about it. The interface package defines and object that describes methods/functions. I don't see why this object can't be associated with a function now that we have function attributes. def foo() pass foo.__impliments__ = FooInterface > * how do we specifry the parameters of classes that do NOT have > associated interfaces? By associating them with an interface. ;) > > Also, type checking in general is good, but what about preconditions (this > > parameter must be an int > 5 < 10) and postconditions and other conditions > > one does now with assertions. Would these be more language extensions in > > your propsal? > > One of my governing philosophies about type systems is that you can > always make them more complex/sophisticated. I don't see that pre and > post-conditions are so important that they need first-class syntax. > Assertions are sufficient. Agreed. I didn't make my point clear though, interface objects can be tagged with any kind of data, like UML. Itamar (a Zope developer) just sent me code that creates pre/postcondition objects that you can tag on an interface. Granted there's no syntax for it, and there shouldn't be, but the interface framework is extensable in that way. That's what I *meant* to say. ;) > To recap: > > How do we do parameter type declarations in scarecrow? There's no syntax, but an API. But wait... maybe there can be the same syntax on both (classes and interfaces), and people can take or leave interfaces and/or class type checking based on their needs. We (DC), for example, badly need type checking on many of our methods that expect input the web. Some sneaky security attacks (and a whole lot of bugs) have been discovered based on passing a different kind of object to a through-the-web method than was expected. Fixing these with assertions was laborous. We would probably go the define-it-in-the-interface type checking because we're going to create interfaces on all our network-exposed components anyway. Others could define their type checking right in the class if they have no need for an interface. > How do we do return type declarations in scarecrow? Same answer as above. I definitely need to read the four (four!?) proposals that exist already. It seems like there are three high level things that were discussed in the type sig: Type Checking Interfaces Type/Class split A pandora's box indeed. Maybe we should take Jim's suggestion and keep them seperate? http://mail.python.org/pipermail/types-sig/1999-December/000250.html -Michel From paulp@ActiveState.com Mon Mar 12 12:41:57 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Mon, 12 Mar 2001 04:41:57 -0800 Subject: [Types-sig] Revive the types sig? References: <3AAC08DB.9D4E96B4@ActiveState.com> Message-ID: <3AACC415.2372BF4C@ActiveState.com> Michael Hudson wrote: > > ... > > Here's mine: don't waste all your energies on discussion here. Spend > at least some of them actually coming up with something concrete and > for bonus points implement it, or we'll just have 1999-12 again. > > Then we can shout about that. I've already started writing, but the shouting will be spectacularly unproductive if we have very different goals and scopes, which is what we had in 1999-12. -- Python: Programming the way Guido indented it. - (originated with Skip Montanaro?) From guido@digicool.com Mon Mar 12 14:00:49 2001 From: guido@digicool.com (Guido van Rossum) Date: Mon, 12 Mar 2001 09:00:49 -0500 Subject: [Types-sig] Re: [Python-Dev] Revive the types sig? In-Reply-To: Your message of "Sun, 11 Mar 2001 21:18:06 PST." References: Message-ID: <200103121400.JAA18715@cj20424-a.reston1.va.home.com> [Responses trimmed] [Ping] > For the record, here is a little idea i came up with on the > last day of the conference: > > Suppose there is a built-in class called "Interface" with the > special property that whenever any immediate descendant of > Interface is sub-classed, we check to make sure all of its > methods are overridden. If any methods are not overridden, > something like InterfaceException is raised. > > This would be sufficient to provide very simple interfaces, > at least in terms of what methods are part of an interface > (it wouldn't do any type checking, but it could go a step > further and check the number of arguments on each method). > > Example: > > >>> class Spam(Interface): > ... def islovely(self): pass > ... > >>> Spam() > TypeError: interfaces cannot be instantiated > >>> class Eggs(Spam): > ... def scramble(self): pass > ... > InterfaceError: class Eggs does not implement interface Spam > >>> class LovelySpam(Spam): > ... def islovely(self): return 1 > ... > >>> LovelySpam() > > > Essentially this would replace the convention of writing a > whole bunch of methods that raise NotImplementedError as a > way of describing an abstract interface, making it a bit easier > to write and causing interfaces to be checked earlier (upon > subclassing, rather than upon method call). > > It should be possible to implement this in Python using metaclasses. You could do this in pure Python using meta-classes. This could go some way towards enforcement of interfaces at the implementation side, but I'm not even sure that that's the real issue. I think enforcement is required more at the call side -- if I expect something that implements interface Foo, I need to be able to efficiently write and execute a type assertion about that. At this point, I'm probably happy to take anything that *says* it implements Foo -- if it doesn't implement some Foo method, I'll find out sooner or later. But that's shallow anyway. The deep(er) part is whether the object passed in *thinks of itself* as implementing the Foo interface. This means that its author has (presumably) spent at least a little time about the invariants that a Foo should obey. The enforcement class you propose does't help there. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@digicool.com Mon Mar 12 14:08:38 2001 From: guido@digicool.com (Guido van Rossum) Date: Mon, 12 Mar 2001 09:08:38 -0500 Subject: [Types-sig] Re: [Python-Dev] Revive the types sig? In-Reply-To: Your message of "Sun, 11 Mar 2001 22:35:28 PST." <3AAC6E30.CFB80A63@ActiveState.com> References: <3AAC6E30.CFB80A63@ActiveState.com> Message-ID: <200103121408.JAA18760@cj20424-a.reston1.va.home.com> > Agreed. There are good reasons for wanting "inline" declarations (ease > of programming and prototyping) and good reasons for wanting separate > interfaces (especially for reuse). I think that this is a strong > argument that we should support both. Just keep in mind that the language shouldn't grow faster than the rate of inflation. Maybe that's why I originally relegated this to Python 3000. :-) > -- > Python: > Programming the way > Guido > indented it. > - (originated with Skip Montanaro?) Did it really? Skip, do you remember? It's a great slogan! We should send you a T-shirt. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@digicool.com Mon Mar 12 14:17:42 2001 From: guido@digicool.com (Guido van Rossum) Date: Mon, 12 Mar 2001 09:17:42 -0500 Subject: [Types-sig] Re: [Python-Dev] Revive the types sig? In-Reply-To: Your message of "Mon, 12 Mar 2001 00:23:53 PST." References: Message-ID: <200103121417.JAA18831@cj20424-a.reston1.va.home.com> > Prelude: > > I've just been digging a bit through the types-sig archives. Aye carumba, > we are opening a big-arsed can of worms up here. > > In a way, we may be leading ourselves down the same path of confusion, > maybe what we need here is a very high level description of vision, scope, > and goals, perchance a use case or two. At DC, we like to do these things > in Wikis so that they don't get lost in the archives. I'd be happy to > create a Wiki for this. Yes, Yes, YES! The goal of whatever it is we're doing here should be to come up with a PEP, and a Wiki seems to be a good way to iteratively approach writing a PEP. (As long as eventually it's moved into the PEP.) I also believe that without an outspoken author who heads the effort, there's no hope. Paul, please either lead the way or say you ain't "it". We've had enough meta discussion. I know *I* ain't gonna be "it" -- I just don't have the time. --Guido van Rossum (home page: http://www.python.org/~guido/) From paulp@ActiveState.com Mon Mar 12 15:06:39 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Mon, 12 Mar 2001 07:06:39 -0800 Subject: [Types-sig] Re: [Python-Dev] Revive the types sig? References: <200103121417.JAA18831@cj20424-a.reston1.va.home.com> Message-ID: <3AACE5FF.8D551F67@ActiveState.com> Guido van Rossum wrote: > > ... > > Yes, Yes, YES! > > The goal of whatever it is we're doing here should be to come up with > a PEP, and a Wiki seems to be a good way to iteratively approach > writing a PEP. (As long as eventually it's moved into the PEP.) I don't know about everybody else but I have a single event queue and that is my inbox. I can use a Wiki if there is a bidirectional mail engine but otherwise I would rather use a mailing list. > I also believe that without an outspoken author who heads the effort, > there's no hope. Paul, please either lead the way or say you ain't > "it". We've had enough meta discussion. I disagree. The problem with previous efforts has been precisely that everybody comes in with radically conflicting goals and approaches. I've "just dived in" twice before with no success. We have a variety of very precise and elaborate documents here: http://www.python.org/sigs/types-sig/ And we are no closer to a final spec. It would be absolutely no use for me to dive in writing another one. That said, I have four new type-related PEPs sitting on my hard drive. But I strongly believe that it is NOT useful to start discussing solutions until we agree on the problem we're trying to solve. I'll release my PEPs, half of the audience will yell: "that doesn't solve the static type checking problem", another (partially overlapping) half will yell: "that isn't as computationally complete as ML" and so forth. -- Python: Programming the way Guido indented it. - (originated with Skip Montanaro?) From othello@javanet.com Mon Mar 12 15:09:00 2001 From: othello@javanet.com (Raymond Hettinger) Date: Mon, 12 Mar 2001 10:09:00 -0500 Subject: [Fwd: [Types-sig] Re: [Python-Dev] Revive the types sig?] Message-ID: <3AACE68C.E0A0BDDB@javanet.com> Ka-Ping Yee wrote: >For the record, here is a little idea i came up with on the >last day of the conference: > >Suppose there is a built-in class called "Interface" with the >special property that whenever any immediate descendant of >Interface is sub-classed, we check to make sure all of its >methods are overridden. If any methods are not overridden, >something like InterfaceException is raised. > >This would be sufficient to provide very simple interfaces, >at least in terms of what methods are part of an interface >(it wouldn't do any type checking, but it could go a step >further and check the number of arguments on each method). > >Example: > > >>> class Spam(Interface): > ... def islovely(self): pass > ... > >>> Spam() > TypeError: interfaces cannot be instantiated > >>> class Eggs(Spam): > ... def scramble(self): pass > ... > InterfaceError: class Eggs does not implement interface Spam > >>> class LovelySpam(Spam): > ... def islovely(self): return 1 > ... > >>> LovelySpam() > > >Essentially this would replace the convention of writing a >whole bunch of methods that raise NotImplementedError as a >way of describing an abstract interface, making it a bit easier >to write and causing interfaces to be checked earlier (upon >subclassing, rather than upon method call). All in all, I REALLY like this approach in terms of simplicity and conformity with the existing language. Also, the approach doesn't require alot of lines of overhead to set-up an interface. One question: Should InterfaceError be a error or a warning? I think the Eggs class should still compile with a warning and allow for islovely() to be added dynanically later. Another suggestion: add a built-in function to dynamically check whether a class meets an interface specs even when the interface was not specified in the class definition. Example: >>> class Sparrow: ... def islovely(self): return 1 >>> s = Sparrow() >>> print implements(s, Spam) 1 One reservation: does this approach provide significant advantages over run-time virtual assertions and normal sub-classing which we already have? Example: >>> class Spam: ... def islovely(self): assert 0, 'Virtual method not implemented' >>> class Eggs(Spam): ... def scramble(self): pass >>> e = Eggs() >>> e.islovely() AssertionError: Virtual method not implemented Submitted for your approval, Raymond Hettinger "In theory, there is no difference between theory and practice. In practice, there is." -- Yogi Berra From guido@digicool.com Mon Mar 12 15:19:45 2001 From: guido@digicool.com (Guido van Rossum) Date: Mon, 12 Mar 2001 10:19:45 -0500 Subject: [Types-sig] Re: [Python-Dev] Revive the types sig? In-Reply-To: Your message of "Mon, 12 Mar 2001 07:06:39 PST." <3AACE5FF.8D551F67@ActiveState.com> References: <200103121417.JAA18831@cj20424-a.reston1.va.home.com> <3AACE5FF.8D551F67@ActiveState.com> Message-ID: <200103121519.KAA19343@cj20424-a.reston1.va.home.com> > > The goal of whatever it is we're doing here should be to come up with > > a PEP, and a Wiki seems to be a good way to iteratively approach > > writing a PEP. (As long as eventually it's moved into the PEP.) > > I don't know about everybody else but I have a single event queue and > that is my inbox. I can use a Wiki if there is a bidirectional mail > engine but otherwise I would rather use a mailing list. There's no mail interface but I strongly recommend giving Wiki a chance. The issue with mail is that it makes it very easy to carry on a discussion, but it's practically impossible to extract the gist of that discussion later from the archives. This means that those people who aren't able to follow the discussion in real time are lost forever. You may try out a hybrid approach: use the Wiki to record progress as it is made in the email discussion; then latecomers (or historians :-) can use the Wiki to get up to speed. Please give this a try! We've tried the email route twice before and it didn't work. > > I also believe that without an outspoken author who heads the effort, > > there's no hope. Paul, please either lead the way or say you ain't > > "it". We've had enough meta discussion. > > I disagree. The problem with previous efforts has been precisely that > everybody comes in with radically conflicting goals and approaches. I've > "just dived in" twice before with no success. We have a variety of very > precise and elaborate documents here: > > http://www.python.org/sigs/types-sig/ > > And we are no closer to a final spec. It would be absolutely no use for > me to dive in writing another one. Not if your thinking hasn't made progress since last year. But you should have had enough feedback and time to think about it to have made up your mind. Really, if you're just inviting people to discuss this till they're on fire, that's exactly what's going to happen, and you might as well give up now. (Saves you & your wife some grief. :-) > That said, I have four new type-related PEPs sitting on my hard drive. > But I strongly believe that it is NOT useful to start discussing > solutions until we agree on the problem we're trying to solve. I'll > release my PEPs, half of the audience will yell: "that doesn't solve the > static type checking problem", another (partially overlapping) half will > yell: "that isn't as computationally complete as ML" and so forth. So be careful to say what your PEPs *do* solve, and tell others that if they have a different problem, that they should look for a different solution, and write theire own damn PEP. Please, you've got to stop this committee-design idea. Take the lead. Propose a design that does what *you* need. Allow yourself to be selfish. See if your critics care enough to write a counter-PEP. If they do, more power to them (nothing is so focusing for a discussion as two opposing proposals). If they don't, their criticism isn't worth much, and you win by default. Now, when all is said and done, you may still end up with an unacceptable proposal, for different reasons. But the first goal here is to come up with a proposal (and possibly a counter-proposal). Focus the discussion! --Guido van Rossum (home page: http://www.python.org/~guido/) From paulp@ActiveState.com Mon Mar 12 16:20:22 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Mon, 12 Mar 2001 08:20:22 -0800 Subject: [Types-sig] Proposed Goals PEP Message-ID: <3AACF746.9A06058F@ActiveState.com> I propose that the first round of the types-sig focus on a syntax for type declarations in classes (and by extension interfaces). The goals of type (recently expanded from three to four) annotations are error checking (err), documentation (doc), optimization (opt) and glue to static type systems (glue). In order to allow the resolution of conflicts between goals I claim that we should treat their order of importance as err, doc, opt and glue. In particular, there are many approaches to opt that would vastly improve Python's performance but vastly degrade its dynamic features. We should agree that Python's strength is that usability comes before performance. Here are the goals expanded: 1. Better error checking. (major) If I call a C function with wrong arguments I get a very readable error message: >>> open(5) Traceback (most recent call last): File "", line 1, in ? TypeError: open, argument 1: expected string, int found If I call a Python function in the standard library with wrong arguments I get this: Traceback (most recent call last): File "", line 1, in ? File "c:\python20\lib\urllib.py", line 61, in urlopen return _urlopener.open(url) File "c:\python20\lib\urllib.py", line 139, in open fullurl = unwrap(fullurl) File "c:\python20\lib\urllib.py", line 857, in unwrap url = string.strip(url) File "c:\python20\lib\string.py", line 80, in strip return s.strip() AttributeError: 'int' object has no attribute 'strip' If there are many levels of code it is not at all clear where the value first went "bad". We want to make it possible to detect the mistake much earlier. The canonical "use case" for this is the standard library. The standard library should check argument values for sanity as much as is is practical. Right now it does not do much checking because the syntax for doing so is obfuscatory and verbose. 2. Better Documentation (major) Right now, it is basically impossible for programs like PyDoc or IDLE to detect the types of arguments to functions. This makes it difficult to give the user a clear summary of what a function expects or returns. Use cases: There should be enough information in Python code so that PyDoc could look at the declaration for the "open" function and tell the usr that the first argument must be a string and the second must be an integer. There should be enough information to hyperlink from references to types to definitions of those types. 3. Optimization (minor) If a Python compiler knew that a function works on integers or strings then it could internally use bytecode or native code optimized for integers or strings. 4. Static type glue (minor) There are a plethora of language-interopability technologies out there and most of them use static type checking systems. Examples include CORBA, COM, DCOM, XPCOM, RMI, .NET, SOAP/WSDL. It would be nice if Python programmers could generate IDL/WSDL/type libraries etc. from type annotations in their Python code. This is even useful in a simple C+Python application. If there are places where C must call into Python, we currently require type conversions to be handled manually. It would be nice if we could use a "reverse SWIG" to generate a wrapper that would make it easy to call Python code without manual type conversions. Constraints: In order to get something done in a reasonable amount of time we must decide which issues not to tackle. At the top of that list, in my opinion, is static type checking (including a formally defined type inferencing system). Second is any form of higher order types. If we could agree that these are NOT on the table we will make progress much more quickly. In general, a major constraint is that a type system for Python should be simple. We should not be afraid to teach the type system to high school students. If Python's type system makes anyone's brain explode then we have not done a good job. -- Python: Programming the way Guido indented it. - (originated with Skip Montanaro?) From qrczak@knm.org.pl Mon Mar 12 16:16:56 2001 From: qrczak@knm.org.pl (Marcin 'Qrczak' Kowalczyk) Date: 12 Mar 2001 16:16:56 GMT Subject: [Types-sig] Re: Revive the types sig? References: Message-ID: Sun, 11 Mar 2001 15:23:07 -0800, Paul Prescod pis= ze: > Even if we took an extreme position and ONLY allowed type annotations > for basic types like strings, numbers and sequences, Python would > still be a better language. I strongly disagree with making it broken-as-designed. A flawed type system adds too much complexity and yields pain in fighting with its limitations compared to benefits. We should either do it such that it really works in most cases, or don't do it at all. When you pass a string instead of a number to a function, the error is often caught immediately as soon as the parameter is used. But when you pass a list and some of its elements don't have the expected type, the error is more subtle. For me the ability to specify the type of an argument, but inability to say anything about elements of a list or tuple, makes the system not worth adding to a language. Similarly, being able to tell something only about builtin types or interfaces. > Java also does not yet have parameterized types and yet it is the > most rapidly growing statically typed programming language! Many complaints about static typing result from the assumption that it must be as poor as in Java, or as complex as in C++. I suggest taking Haskell, ML or Eiffel as inspiration, not Java. Java requires type casts to overcome limitations of the static type system, because of lack of genericity (besides builtin array hack) and lack of the ability to form conjunctions of interfaces. Even Java is going to add genericity, and there already exist Java extensions which do it. Lack of genericity is IMHO the biggest single flaw in Java's type system. > Deferred for future versions (or never): >=20 > * compile-time type checking > * parameterized types > * declarations for variables and attributes I'm afraid that it would give python the worst parts of static and dynamic typing. I agree that Python's typing should be optional, but disagree with the rest of your goals :-( --=20 __("< Marcin Kowalczyk * qrczak@knm.org.pl http://qrczak.ids.net.pl/ \__/ ^^ SYGNATURA ZAST=CAPCZA QRCZAK From othello@javanet.com Mon Mar 12 16:42:49 2001 From: othello@javanet.com (Raymond Hettinger) Date: Mon, 12 Mar 2001 11:42:49 -0500 Subject: [Types-sig] Proposed Goals PEP References: <3AACF746.9A06058F@ActiveState.com> Message-ID: <3AACFC89.875EE423@javanet.com> Paul Prescod wrote: > In order to get something done in a reasonable amount of time we must > decide which issues not to tackle. At the top of that list, in my > opinion, is static type checking (including a formally defined type > inferencing system). Second is any form of higher order types. If we > could agree that these are NOT on the table we will make progress much > more quickly. Agreed. Higher order types are for languages like ML which a designed around strong typing. > > In general, a major constraint is that a type system for Python should > be simple. We should not be afraid to teach the type system to high > school students. If Python's type system makes anyone's brain explode > then we have not done a good job. > Well said. Think minimally! Raymond Hettinger From paulp@ActiveState.com Mon Mar 12 16:45:12 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Mon, 12 Mar 2001 08:45:12 -0800 Subject: [Types-sig] Re: Revive the types sig? References: Message-ID: <3AACFD18.E4CE0115@ActiveState.com> Disclaimer: I am not the types-czar. I will design one system with the help of people who are like-minded. You can design another with different goals if you like. Guido can choose. Marcin 'Qrczak' Kowalczyk wrote: > > ... > > I strongly disagree with making it broken-as-designed. A flawed type > system adds too much complexity and yields pain in fighting with its > limitations compared to benefits. We should either do it such that > it really works in most cases, or don't do it at all. As a thought experiment, let's imagine the radically simple situation I outlined where you can only type check strings and integers. Please explain how this will "yield pain in fighting with its limitations?" "Simple" type systems are painful in statically typed languages because there is no way to get around them. In Python, you can just ignore them when they don't do what you need. > When you pass a string instead of a number to a function, the error > is often caught immediately as soon as the parameter is used. Often the parameter is not used for a long, long time. Compare the traceback from open(None) to the one from urllib.urlopen(None). > But when > you pass a list and some of its elements don't have the expected type, > the error is more subtle. Yes, well. When you pass a list of integers that you expect to be prime and some aren't prime, that's subtle too. We can't save the world. We can only make a type system that is simple enough for people to like and use. > For me the ability to specify the type of an argument, but inability to > say anything about elements of a list or tuple, makes the system not > worth adding to a language. Similarly, being able to tell something > only about builtin types or interfaces. I'm not saying that we will be that draconian but if the design spins out of control we might end up there. > Many complaints about static typing result from the assumption that > it must be as poor as in Java, or as complex as in C++. > I suggest taking Haskell, ML or Eiffel as inspiration, not Java. Well, I disliked ML's type system immensely and Eiffel's is so complex that the Eiffel FAQ admits that most compilers do not implement it in its entirety. Many people (myself included) consider ML to be rocket science. I haven't used Haskell but compared to what I envision I know it also has a brain-exploding type system. Monads??? "Haskell is a typeful programming language: (A phrase due to Luca Cardelli.) Types are pervasive, and the newcomer is best off becoming well-aware of the full power and complexity of Haskell's type system from the outset." - http://www.cs.yale.edu/haskell/tutorial.html Python is not a typeful programming language. It should never become one. > Java requires type casts to overcome limitations of the static type > system, because of lack of genericity (besides builtin array hack) > and lack of the ability to form conjunctions of interfaces. Maybe this simplicity is why Java is more popular than C++ > Even Java is going to add genericity, and there already exist Java > extensions which do it. Lack of genericity is IMHO the biggest single > flaw in Java's type system. As a computer scientist and sometimes Java programmer, I agree wholeheartedly. But IMHO, Python cannot and will not do static type checking better than Java and still remain Python. We have much more modest goals. Nicer error messages. Better documentation. The ability to auto-generate IDL. A little bit faster handling of ints, strings and maybe (maybe!) method tables. I mean Python is a language that is so radically simple that it doesn't even have a concept of protected or truly private methods! -- Python: Programming the way Guido indented it. - (originated with Skip Montanaro?) From Samuele Pedroni Mon Mar 12 16:47:22 2001 From: Samuele Pedroni (Samuele Pedroni) Date: Mon, 12 Mar 2001 17:47:22 +0100 (MET) Subject: [Types-sig] about sparse inputs from the jython userbase & types, language extensions Message-ID: <200103121647.RAA15331@core.inf.ethz.ch> Hi. What follows is maybe to abstract or naive to be useful, if reading this is waste of time: sorry. Further I ignore the content of the P3K kick-start session... "We" are planning to add many features to python. It has also been explicitly written that this is for the developers to have fun too ;). Exact arithmetic, behind the scene promotion on overflow, etc... nested scopes, iterators A bit joking: lim(t->oo) python ~ Common Lisp Ok, in python programs and data are not that much the same, we don't have CL macros (but AFAIK dylan is an example of a language without data&programs having the same structure but with CL-like macros , so maybe...), and "we" are not as masochistic as a commitee can be, and we have not the all the history that CL should carry. Python does not have (by now) optional static typing (CL has such a beast, everybody knows), but this is always haunting around, mainly for documentation and error checking purposes. Many of the proposals also go in the direction of making life easier for newbies, even for programming newbies... (this is not a paradox, a regular and well chosen subset of CL can be appopriate for them and the world knows a beast called scheme). Joke: making newbie happy is dangerous, then they will never want to learn C ;) The point: what is some (sparse) part of jython user base asking for? 1. better java intergration (for sure). 2. p-e-r-f-o-r-m-a-n-c-e They ask why is jython so slow, why it does not exploit unboxed int or float (the more informed one), whether it is not possible to translate jython to java achieving performance... The python answer about performance is: - Think, you don't really need it, - find the hotspot and code it in C, - programmer speed is more important than pure program speed, - python is just a glue language Jython one is not that different. If someone comes from C or much java this is fair. For the happy newbie that's deceiving. (And can become frustrating even for the experienced open-source programmer that wants to do more in less time: be able to do as much things as possible in python would be nice ). If python importance will increase IMHO this will become a real issue (also from java, people is always asking for more performance). Let some software house give them the right amount of perfomance and dynamism out of python for $xK (that what happens nowaday with CL), even more deceiving. (I'm aware that dealing with that, also from a purely code complexity viewpoint, may be too much for an open project in term of motivation) regards, Samuele Pedroni. PS: I'm aware of enough theoretical approaches to performance to know that optional typing is just one of the possible, the point is that performance as an issue should not be underestimated. From paulp@ActiveState.com Mon Mar 12 16:50:58 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Mon, 12 Mar 2001 08:50:58 -0800 Subject: [Types-sig] Re: [Python-Dev] Revive the types sig? References: <200103121417.JAA18831@cj20424-a.reston1.va.home.com> <3AACE5FF.8D551F67@ActiveState.com> <200103121519.KAA19343@cj20424-a.reston1.va.home.com> Message-ID: <3AACFE72.36A6BE49@ActiveState.com> Guido van Rossum wrote: > > ... > > You may try out a hybrid approach: use the Wiki to record progress as > it is made in the email discussion; then latecomers (or historians :-) > can use the Wiki to get up to speed. That would be fine with me. Every time we update the document we could record why we did so in the Wiki. Michel, could you please set up a Wiki for the goals and requirements document? -- Python: Programming the way Guido indented it. - (originated with Skip Montanaro?) From guido@digicool.com Mon Mar 12 16:47:54 2001 From: guido@digicool.com (Guido van Rossum) Date: Mon, 12 Mar 2001 11:47:54 -0500 Subject: [Types-sig] Re: Revive the types sig? In-Reply-To: Your message of "12 Mar 2001 16:16:56 GMT." References: Message-ID: <200103121647.LAA20753@cj20424-a.reston1.va.home.com> [Paul] > > Even if we took an extreme position and ONLY allowed type annotations > > for basic types like strings, numbers and sequences, Python would > > still be a better language. [Marcin 'Qrczak' Kowalczyk] > I strongly disagree with making it broken-as-designed. A flawed type > system adds too much complexity and yields pain in fighting with its > limitations compared to benefits. We should either do it such that > it really works in most cases, or don't do it at all. > > When you pass a string instead of a number to a function, the error > is often caught immediately as soon as the parameter is used. But when > you pass a list and some of its elements don't have the expected type, > the error is more subtle. > > For me the ability to specify the type of an argument, but inability to > say anything about elements of a list or tuple, makes the system not > worth adding to a language. Similarly, being able to tell something > only about builtin types or interfaces. > > > Java also does not yet have parameterized types and yet it is the > > most rapidly growing statically typed programming language! > > Many complaints about static typing result from the assumption that > it must be as poor as in Java, or as complex as in C++. > > I suggest taking Haskell, ML or Eiffel as inspiration, not Java. > Java requires type casts to overcome limitations of the static type > system, because of lack of genericity (besides builtin array hack) > and lack of the ability to form conjunctions of interfaces. > > Even Java is going to add genericity, and there already exist Java > extensions which do it. Lack of genericity is IMHO the biggest single > flaw in Java's type system. > > > Deferred for future versions (or never): > > > > * compile-time type checking > > * parameterized types > > * declarations for variables and attributes > > I'm afraid that it would give python the worst parts of static and > dynamic typing. > > I agree that Python's typing should be optional, but disagree with > the rest of your goals :-( Marcin, Your comments ask for a "classic" type system as found in other statically typed languages. You're giving all the right examples. Paul has made it clear that he's given up on that, and wants to do something else. Given that we've tried to come up with a classic type system twice before and failed, I have to agree with the process here: let's try something completely different. Let it be a spectacular success or a valiant failure, but let's not bang our heads against the same wall for the third time. Clearly you and Paul don't agree on what the goal is. I suggest that rather than wasting your time by arguing with Paul, you should go off and propose your own PEP, that furthers *your* goals. You can do that in the same types-SIG -- the charter is wide enough. Just make it clear that you are barking up a different tree. I expect that Jeremy Hylton will support your efforts, and who knows who else. There's nothing wrong with writing two PEPs to choose from! It makes a lot more sense to me than trying to unify into one proposal what are clearly very different goals. --Guido van Rossum (home page: http://www.python.org/~guido/) From neelk@cswcasa.com Mon Mar 12 16:57:03 2001 From: neelk@cswcasa.com (Krishnaswami, Neel) Date: Mon, 12 Mar 2001 11:57:03 -0500 Subject: [Types-sig] Proposed Goals PEP Message-ID: Paul Prescod [mailto:paulp@ActiveState.com] wrote: > > I propose that the first round of the types-sig focus on a syntax for > type declarations in classes (and by extension interfaces). > > Here are the goals expanded: > > 1. Better error checking. (major) > If there are many levels of code it is not at all clear where > the value first went "bad". We want to make it possible to detect > the mistake much earlier. > > The canonical "use case" for this is the standard library. > The standard library should check argument values for sanity as > much as is is practical. Right now it does not do much checking > because the syntax for doing so is obfuscatory and verbose. Agreed. This is the big one. > 2. Better Documentation (major) > > Right now, it is basically impossible for programs like PyDoc > or IDLE to detect the types of arguments to functions. This makes > it difficult to give the user a clear summary of what a function > expects or returns. This isn't going to help unless you have parametrized types. The most common case of type error in my code is when I pass in a list or dictionary with bogus element types. This is because usually a general list or dictionary of a particular shape is being used as a custom "type." Errors with incorrectly handling primitive types just doesn't seem to happen to me -- even the string/sequence thing basically never bites me. I would be interested in other peoples' experiences, since I would be surprised > 3. Optimization (minor) > > If a Python compiler knew that a function works on integers or strings > then it could internally use bytecode or native code optimized for > integers or strings. This is a totally unrealistic expectation. Adding typechecking is going to make typed Python code slower, not faster, for a very long time. This is for two reasons. One, the number of typechecks that the interpreter does will go up, and without an optimization pass (which is error prone to write) they can't be removed. Two, much of the optimization will be from improving the representation of objects, and that would require major surgery to the C FFI. This should not be a 1.0 goal -at all-, except in the modest sense that typed code shouldn't run too much more slowly than ordinary Python code. If you want to improve performance, focus on improving performance as a *separate* task -- stealing Scheme interpreter implementation tricks can win a 2-5x speedup without changing Python's semantics at all. So the order should be err, doc, glue, with no mention of opt whatsoever. > In order to get something done in a reasonable amount of time we must > decide which issues not to tackle. At the top of that list, in my > opinion, is static type checking (including a formally defined type > inferencing system). Second is any form of higher order types. If we > could agree that these are NOT on the table we will make progress much > more quickly. Type inference in the presence of subtyping is still generating research papers, which should scare you a lot. Even static typechecking is also unfeasible with the current semantics of Python, becauseit would require substantial partial evaluation (calculate which clases are live and what their methods are), which is also a subject generating research papers. Therefore, both of these should be off-limits. However, without parametric types there's really no gain from type declarations -- there's nothing that you can't do today with NotImplementedError. > In general, a major constraint is that a type system for Python should > be simple. We should not be afraid to teach the type system to high > school students. If Python's type system makes anyone's brain explode > then we have not done a good job. Yep. -- Neel Krishnaswami neelk@cswcasa.com From michel@digicool.com Mon Mar 12 17:00:26 2001 From: michel@digicool.com (Michel Pelletier) Date: Mon, 12 Mar 2001 09:00:26 -0800 (PST) Subject: [Types-sig] Re: [Python-Dev] Revive the types sig? In-Reply-To: <3AACFE72.36A6BE49@ActiveState.com> Message-ID: On Mon, 12 Mar 2001, Paul Prescod wrote: > Guido van Rossum wrote: > > > > ... > > > > You may try out a hybrid approach: use the Wiki to record progress as > > it is made in the email discussion; then latecomers (or historians :-) > > can use the Wiki to get up to speed. > > That would be fine with me. Every time we update the document we could > record why we did so in the Wiki. Michel, could you please set up a Wiki > for the goals and requirements document? Sure. http://www.zope.org/Members/michel/types-sig/FrontPage You'll need a (free) Zope.org account to edit the Wiki. I can also make it so that any anonymous visitor can edit it, that might be a bit too wide open for you, it does give me the willies. The text formating rules are called StructuredText, you may know them, you may not. If you don't, you can select 'plaintext' as your format and no special markup sniffing will be done. The Wiki is also self documenting, and there's a sandbox you can play in to get the hang of it. Good luck! -Michel From tim.hochberg@ieee.org Mon Mar 12 17:34:44 2001 From: tim.hochberg@ieee.org (Tim Hochberg) Date: Mon, 12 Mar 2001 10:34:44 -0700 Subject: [Types-sig] Proposed Goals PEP References: <3AACF746.9A06058F@ActiveState.com> Message-ID: <175b01c0ab1a$ba638e80$87740918@CX781526B> > Constraints: > > In order to get something done in a reasonable amount of time we must > decide which issues not to tackle. At the top of that list, in my > opinion, is static type checking (including a formally defined type > inferencing system). Second is any form of higher order types. If we > could agree that these are NOT on the table we will make progress much > more quickly. > > In general, a major constraint is that a type system for Python should > be simple. We should not be afraid to teach the type system to high > school students. If Python's type system makes anyone's brain explode > then we have not done a good job. First off: thanks for getting this going again Paul; I know it's a thankless job. Before things get too set in stone (or too chaotic), let me throw at a slightly different approach. That is to provide the machinery for type checking with essentially no formal type system at all. THE BASICS Consider this code (syntax borrowed from one of Guido's old proposals): def spam(eggs : Integer, beans : File) -> String: #.... return x This would be compiled to: def spam(eggs, bakedBeans): if __typecheck__ and not Integer.check(eggs): raise TypeError("Informative message here") if __typecheck__ and not File.check(beans): raise TypeError("Informative message here") # ... if __typecheck__ and not String.check(x): raise TypeError("Informative message here") return x Note that Integer, File and String are all bound when the def is executed, just like default argument are now. Also, __typecheck__ is magic variable similar to debug so that typechecks can be omitted completely during compilation so that there is no speed penalty when compiled this way. The above has the great strength that it's easy to explain! I also believe that it can support a rich, runtime type system (err) and is useful for doc as well. It's utility may well be limited for opt and glue though. In addition to the above syntax, _some_ type support would be included. This would consist of typecheck objects for the basic types (int, string, complex, etc). One possibility would be to add this information to the TypeObjects in the types file, but for the moment I'm going to assume that they would a separate object. Consider Integer as rendered in Python: class Integer: "A builtin, machine size integer" name = "Integer" def check(self, object): return isinstance(object, types.IntType) Integer = Integer() Note that it has a docstring and a name field, documentation extraction tools could extract this information for documention purposes. That's not so strong an argument for integers, but useful for more complicated types. For the time being I'll assume that this information lives in the typecheck module. Already this is provides simple type checking fot the basic types, but we're just getting started.... PARAMETERIZED TYPES Parameterized types are easy to support in this model, although the checks are potentially very expensive. Consider a parameterized list type: class ListOf: "A parameterized list of objects" def __init__(self, type): self.type = type self.name = "ListOf(%)" % type.name def check(self, object): if not isinstance(object, types.ListType): return 0 for item in object: if not self.type.check(item): return 0 return 1 Which could be used as: from typecheck import Integer, ListOf def integralSum(l : ListOf(Integer)) -> Integer: "Return the sum of the integers in l" # .... COMPOSITE and CUSTOM TYPES One can easily define composite types in this framework as well. I'm not sure how far we should directly support this sort of stuff at first, but it's comforting to know that the possibility is there. Here's an example of a generic custom type: class CompositeOf: "A composite type" def __init__(self, *types): self.types = types self.name = "CompositeOf(%s)" % ", ".join([t.name for t in types]) def check(self, object): for t in types: if types.check(t): return 1 return 0 Used like: from typecheck import Integer, String, CompositeOf def iTakeAnIntOrAString(x : CompositeOf(Integer, String)): #... Custom types such as odd integers would also be easy to define. The way that these more complicated types fall out of a simple proposal makes this feel very pythonic to me. INTERFACES I'm guilty of not looking at the interface proposals in any detail, but presumably this proposal and/or interfaces could be munged so that interfaces would be acceptable type parameters. One approach would be just to give interfaces a check method. SUMMARY This idea looks like a good starting point to me. It's simple, it can express complex types, it helps both err and doc. On the downside, it's unlikely to help opt, and I don't know about glue. Regards, -tim From klm@digicool.com Mon Mar 12 18:02:10 2001 From: klm@digicool.com (Ken Manheimer) Date: Mon, 12 Mar 2001 13:02:10 -0500 (EST) Subject: [Types-sig] Re: [Python-Dev] Revive the types sig? Message-ID: I included a bit or orienting text in the page. I'm willing to lend a hand, if people need help getting acquainted with them (though i may start with suggesting reading the HelpPage). Ken klm@digicool.com Date: Mon, 12 Mar 2001 09:00:26 -0800 (PST) From: Michel Pelletier To: Paul Prescod cc: Subject: Re: [Types-sig] Re: [Python-Dev] Revive the types sig? On Mon, 12 Mar 2001, Paul Prescod wrote: > Guido van Rossum wrote: > > > > ... > > > > You may try out a hybrid approach: use the Wiki to record progress as > > it is made in the email discussion; then latecomers (or historians :-) > > can use the Wiki to get up to speed. > > That would be fine with me. Every time we update the document we could > record why we did so in the Wiki. Michel, could you please set up a Wiki > for the goals and requirements document? Sure. http://www.zope.org/Members/michel/types-sig/FrontPage You'll need a (free) Zope.org account to edit the Wiki. I can also make it so that any anonymous visitor can edit it, that might be a bit too wide open for you, it does give me the willies. The text formating rules are called StructuredText, you may know them, you may not. If you don't, you can select 'plaintext' as your format and no special markup sniffing will be done. The Wiki is also self documenting, and there's a sandbox you can play in to get the hang of it. Good luck! From jeremy@alum.mit.edu Mon Mar 12 02:25:57 2001 From: jeremy@alum.mit.edu (Jeremy Hylton) Date: Sun, 11 Mar 2001 21:25:57 -0500 (EST) Subject: [Types-sig] Proposed Goals PEP In-Reply-To: References: Message-ID: <15020.13237.509962.828374@w221.z064000254.bwi-md.dsl.cnc.net> >>>>> "NK" == Krishnaswami, Neel writes: >> 2. Better Documentation (major) >> >> Right now, it is basically impossible for programs like PyDoc or >> IDLE to detect the types of arguments to functions. This makes it >> difficult to give the user a clear summary of what a function >> expects or returns. NK> This isn't going to help unless you have parametrized types. The NK> most common case of type error in my code is when I pass in a NK> list or dictionary with bogus element types. This is because NK> usually a general list or dictionary of a particular shape is NK> being used as a custom "type." Errors with incorrectly handling NK> primitive types just doesn't seem to happen to me -- even the NK> string/sequence thing basically never bites me. I agree that problems with container elements is a common problem. In the absence of a system with parameterized types, it appears that we declare lots of "any"s and add a cast mechanism. Yuck. >> 3. Optimization (minor) >> >> If a Python compiler knew that a function works on integers or >> strings then it could internally use bytecode or native code >> optimized for integers or strings. NK> This is a totally unrealistic expectation. Adding typechecking NK> is going to make typed Python code slower, not faster, for a NK> very long time. I agree with you here, too. I'd be interested to hear how much of a slowdown in Python runtime speed is acceptable to support error checking and/or documentation. NK> This should not be a 1.0 goal -at all-, except in the modest NK> sense that typed code shouldn't run too much more slowly than NK> ordinary Python code. If you want to improve performance, focus NK> on improving performance as a *separate* task -- stealing Scheme NK> interpreter implementation tricks can win a 2-5x speedup without NK> changing Python's semantics at all. A 2-5x improvement sounds bigger than I would expect. What Scheme implementation techniques did you have in mind? Jeremy From jeremy@alum.mit.edu Mon Mar 12 02:31:26 2001 From: jeremy@alum.mit.edu (Jeremy Hylton) Date: Sun, 11 Mar 2001 21:31:26 -0500 (EST) Subject: [Types-sig] Re: [Python-Dev] Revive the types sig? In-Reply-To: <200103120711.AAA09711@localhost.localdomain> References: <15020.9404.557943.164934@w221.z064000254.bwi-md.dsl.cnc.net> <200103120711.AAA09711@localhost.localdomain> Message-ID: <15020.13566.636415.914870@w221.z064000254.bwi-md.dsl.cnc.net> >>>>> "UO" == Uche Ogbuji writes: UO> Jeremy Hylton: >> If documentation is really the problem you want to solve, I >> imagine we'd make much more progress if we could agree on a >> javadoc-style format for documentation. The ability to add >> return-type declarations to functions and methods doesn't seem >> like much of a win. UO> If I understand Jeremy correctly, I am in strong agreement that UO> it is at least worth trying the structured documentation UO> approach to signalling pre- and post-conditions before turning UO> Python into a rather different language. I was being a bit contrary :-). If the goal of a type system is to declare that some functions return an int and others return an object, I suspect that good documentation will be more valuable to more people. (And it would have less runtime cost than type checking!) The Python community, however, has been trying to come up with a simple and sufficient documentation system for far longer than people have been worrying about its type system. I'm ambivalent about static typing, although I agree with Marcin and Neel that we know enough about bad static type systems (C++ and Java) that we ought to know enough to include parameterized types. It sounds like those of us in this camp will have to come up with our own proposal. Jeremy From danwang@CS.Princeton.EDU Mon Mar 12 18:40:31 2001 From: danwang@CS.Princeton.EDU (Daniel Wang) Date: 12 Mar 2001 13:40:31 -0500 Subject: [Types-sig] Re: Revive the types sig? In-Reply-To: Paul Prescod's message of "Mon, 12 Mar 2001 08:45:12 -0800" References: <3AACFD18.E4CE0115@ActiveState.com> Message-ID: Paul Prescod writes: > Disclaimer: I am not the types-czar. I will design one system with the > help of people who are like-minded. You can design another with > different goals if you like. Guido can choose. > Before you give up the ship on static types... are you aware of the work by Henglein and Rehof? Safe Polymorphic Type Inference for a Dynamically Typed Language: Translating Scheme to ML. Proceedings FPCA '95, ACM SIGPLAN-SIGARCH Conference on Functional Programming Languages and Computer Architecture, La Jolla, California, June 1995. We describe a new method for polymorphic type inference for the dynamically typed language Scheme. The method infers both types and explicit run-time type operations (coercions) for a given program. It can be used to statically debug Scheme programs and to give a high-level translation to ML, in essence providing an ``embedding'' of Scheme into ML. http://www.research.microsoft.com/~rehof/fpca95.ps Perhaps, there's a bit too much type-theory for most readers. Everyone, should get past the first page or two. But it basically outlines an approch that lets dynamic typing and static typing to coexist relatively painlessly. A good deal of the papers talks about how todo inference, but you can probably just ignore inference and use some of the ideas there for an explicitly typed system, for which a type inference engine could be built later. If people, are utterly confused I can attempt to decode specific questions about the paper. From barry@digicool.com Mon Mar 12 23:15:15 2001 From: barry@digicool.com (Barry A. Warsaw) Date: Mon, 12 Mar 2001 18:15:15 -0500 Subject: [Types-sig] Re: [Python-Dev] Revive the types sig? References: <15020.9404.557943.164934@w221.z064000254.bwi-md.dsl.cnc.net> <200103120711.AAA09711@localhost.localdomain> Message-ID: <15021.22659.616556.298360@anthem.wooz.org> >>>>> "UO" == Uche Ogbuji writes: UO> I know this isn't the types SIG and all, but since it has come UO> up here, I'd like to (once again) express my violent UO> disagreement with the efforts to add static typing to Python. UO> After this, I won't pursue the thread further here. Thank you Uche! I couldn't agree more, and will also try to follow your example, at least until we see much more concrete proposals from the types-sig. I just want to make a few comments for the record. First, it seemed to me that the greatest push for static type annotations at IPC9 was from the folks implementing Python on top of frameworks other than C. I know from my own experiences that there is the allure of improved performance, e.g. JPython, given type hints available to the compiler. While perhaps a laudable goal, this doesn't seem to be a stated top priority of Paul's. Second, if type annotations are to be seriously considered for inclusion in Python, I think we as a community need considerable experience with a working implementation. Yes, we need PEPs and specs and such, but we need something real and complete that we can play with, /without/ having to commit to its acceptance in mainstream Python. Therefore, I think it'll be very important for type annotation proponents to figure out a way to allow people to see and play with an implementation in an experimental way. This might mean an extensive set of patches, a la Stackless. After seeing and talking to Neil and Andrew about PTL and Quixote, I think there might be another way. It seems that their approach might serve as a framework for experimental Python syntaxes with minimal overhead. If I understand their work correctly, they have their own compiler which is built on Jeremy's tools, and which accepts a modified Python grammar, generating different but compatible bytecode sequences. E.g., their syntax has a "template" keyword approximately equivalent to "def" and they do something different with bare strings left on the stack. The key trick is that it all hooks together with an import hook so normal Python code doesn't need to know anything about the mechanics of PTL compilation. Given a homepage.ptl file, they just do an "import homepage" and this gets magically transformed into a .ptlc file and normal Python objects. If I've got this correct, it seems like it would be a powerful tool for playing with alternative Python syntaxes. Ideally, the same technique would allow the types-sig folks to create a working implementation that would require only the installation of an import hook. This would let them build their systems with type annotation and prove to the skeptical among us of their overwhelming benefit. Cheers, -Barry From guido@digicool.com Mon Mar 12 23:41:01 2001 From: guido@digicool.com (Guido van Rossum) Date: Mon, 12 Mar 2001 18:41:01 -0500 Subject: [Types-sig] Re: [Python-Dev] Revive the types sig? In-Reply-To: Your message of "Mon, 12 Mar 2001 18:15:15 EST." <15021.22659.616556.298360@anthem.wooz.org> References: <15020.9404.557943.164934@w221.z064000254.bwi-md.dsl.cnc.net> <200103120711.AAA09711@localhost.localdomain> <15021.22659.616556.298360@anthem.wooz.org> Message-ID: <200103122341.SAA23054@cj20424-a.reston1.va.home.com> > >>>>> "UO" == Uche Ogbuji writes: > > UO> I know this isn't the types SIG and all, but since it has come > UO> up here, I'd like to (once again) express my violent > UO> disagreement with the efforts to add static typing to Python. > UO> After this, I won't pursue the thread further here. > > Thank you Uche! I couldn't agree more, and will also try to follow > your example, at least until we see much more concrete proposals from > the types-sig. I just want to make a few comments for the record. Barry, you were supposed to throw a brick at me with this content at the meeting, on Eric's behalf. Why didn't you? I was waiting for someone to explain why this was a big idea, but everybody kept their face shut! :-( > First, it seemed to me that the greatest push for static type > annotations at IPC9 was from the folks implementing Python on top of > frameworks other than C. I know from my own experiences that there is > the allure of improved performance, e.g. JPython, given type hints > available to the compiler. While perhaps a laudable goal, this > doesn't seem to be a stated top priority of Paul's. > > Second, if type annotations are to be seriously considered for > inclusion in Python, I think we as a community need considerable > experience with a working implementation. Yes, we need PEPs and specs > and such, but we need something real and complete that we can play > with, /without/ having to commit to its acceptance in mainstream > Python. Therefore, I think it'll be very important for type > annotation proponents to figure out a way to allow people to see and > play with an implementation in an experimental way. +1 > This might mean an extensive set of patches, a la Stackless. After > seeing and talking to Neil and Andrew about PTL and Quixote, I think > there might be another way. It seems that their approach might serve > as a framework for experimental Python syntaxes with minimal overhead. > If I understand their work correctly, they have their own compiler > which is built on Jeremy's tools, and which accepts a modified Python > grammar, generating different but compatible bytecode sequences. > E.g., their syntax has a "template" keyword approximately equivalent > to "def" and they do something different with bare strings left on the > stack. I'm not sure this is viable. I believe Jeremy's compiler package actually doesn't have its own parser -- it uses the parser module (which invokes Python's standard parse) and then transmogrifies the parse tree into something more usable, but it doesn't change the syntax! Quixote can get away with this because their only change is giving a different meaning to stand-alone string literals. But for type annotations this doesn't give enough freedom, I expect. > The key trick is that it all hooks together with an import hook so > normal Python code doesn't need to know anything about the mechanics > of PTL compilation. Given a homepage.ptl file, they just do an > "import homepage" and this gets magically transformed into a .ptlc > file and normal Python objects. That would be nice, indeed. > If I've got this correct, it seems like it would be a powerful tool > for playing with alternative Python syntaxes. Ideally, the same > technique would allow the types-sig folks to create a working > implementation that would require only the installation of an import > hook. This would let them build their systems with type annotation > and prove to the skeptical among us of their overwhelming benefit. +1 --Guido van Rossum (home page: http://www.python.org/~guido/) From jeremy@alum.mit.edu Mon Mar 12 23:44:14 2001 From: jeremy@alum.mit.edu (Jeremy Hylton) Date: Mon, 12 Mar 2001 18:44:14 -0500 (EST) Subject: [Types-sig] Re: [Python-Dev] Revive the types sig? In-Reply-To: <200103122341.SAA23054@cj20424-a.reston1.va.home.com> References: <15020.9404.557943.164934@w221.z064000254.bwi-md.dsl.cnc.net> <200103120711.AAA09711@localhost.localdomain> <15021.22659.616556.298360@anthem.wooz.org> <200103122341.SAA23054@cj20424-a.reston1.va.home.com> Message-ID: <15021.24398.737816.108344@w221.z064000254.bwi-md.dsl.cnc.net> >>>>> "GvR" == Guido van Rossum writes: >> syntaxes with minimal overhead. If I understand their work >> correctly, they have their own compiler which is built on >> Jeremy's tools, and which accepts a modified Python grammar, >> generating different but compatible bytecode sequences. E.g., >> their syntax has a "template" keyword approximately equivalent to >> "def" and they do something different with bare strings left on >> the stack. GvR> I'm not sure this is viable. I believe Jeremy's compiler GvR> package actually doesn't have its own parser -- it uses the GvR> parser module (which invokes Python's standard parse) and then GvR> transmogrifies the parse tree into something more usable, but GvR> it doesn't change the syntax! Quixote can get away with this GvR> because their only change is giving a different meaning to GvR> stand-alone string literals. But for type annotations this GvR> doesn't give enough freedom, I expect. Right! It would be nice to kill two birds with one stone though. We could generate an alternate parser for Python and have it generate the same AST, decorated with type information. Even if the type experiment doesn't work out, the alternate parser might be useful. Jeremy From barry@digicool.com Mon Mar 12 23:52:57 2001 From: barry@digicool.com (Barry A. Warsaw) Date: Mon, 12 Mar 2001 18:52:57 -0500 Subject: [Types-sig] Re: [Python-Dev] Revive the types sig? References: <15020.9404.557943.164934@w221.z064000254.bwi-md.dsl.cnc.net> <200103120711.AAA09711@localhost.localdomain> <15021.22659.616556.298360@anthem.wooz.org> <200103122341.SAA23054@cj20424-a.reston1.va.home.com> Message-ID: <15021.24921.998693.156809@anthem.wooz.org> >>>>> "GvR" == Guido van Rossum writes: GvR> Barry, you were supposed to throw a brick at me with this GvR> content at the meeting, on Eric's behalf. Why didn't you? I GvR> was waiting for someone to explain why this was a big idea, GvR> but everybody kept their face shut! :-( I actually thought I had, but maybe it was a brick made of bouncy spam instead of concrete. :/ GvR> I'm not sure this is viable. I believe Jeremy's compiler GvR> package actually doesn't have its own parser -- it uses the GvR> parser module (which invokes Python's standard parse) and GvR> then transmogrifies the parse tree into something more GvR> usable, but it doesn't change the syntax! Quixote can get GvR> away with this because their only change is giving a GvR> different meaning to stand-alone string literals. But for GvR> type annotations this doesn't give enough freedom, I expect. I thought PTL definitely included a "template" declaration keyword, a la, def, so they must have some solution here. MEMs guys? -Barry From nas@arctrix.com Tue Mar 13 00:07:30 2001 From: nas@arctrix.com (Neil Schemenauer) Date: Mon, 12 Mar 2001 16:07:30 -0800 Subject: [Types-sig] parsers and import hooks [Was: Revive the types sig?] In-Reply-To: <200103122341.SAA23054@cj20424-a.reston1.va.home.com>; from guido@digicool.com on Mon, Mar 12, 2001 at 06:41:01PM -0500 References: <15020.9404.557943.164934@w221.z064000254.bwi-md.dsl.cnc.net> <200103120711.AAA09711@localhost.localdomain> <15021.22659.616556.298360@anthem.wooz.org> <200103122341.SAA23054@cj20424-a.reston1.va.home.com> Message-ID: <20010312160729.A2976@glacier.fnational.com> [Recipient addresses brutally slashed.] On Mon, Mar 12, 2001 at 06:41:01PM -0500, Guido van Rossum wrote: > I'm not sure this is viable. I believe Jeremy's compiler package > actually doesn't have its own parser -- it uses the parser module > (which invokes Python's standard parse) and then transmogrifies the > parse tree into something more usable, but it doesn't change the > syntax! Yup. Having a more flexible Python-like parser would be cool but I don't think I'd ever try to implement it. I know Christian Tismer wants one. Maybe he will volunteer. :-) [On using import hooks to load modules with modified syntax/semantics] > That would be nice, indeed. Its nice if you can get it to work. import hooks are a bitch to write and are slow. Also, you get trackbacks from hell. It would be nice if there were higher level hooks in the interpreter. imputil.py did no do the trick for me after wrestling with it for hours. Neil From nkauer@users.sourceforge.net Tue Mar 13 00:09:10 2001 From: nkauer@users.sourceforge.net (Nikolas Kauer) Date: Mon, 12 Mar 2001 18:09:10 -0600 (CST) Subject: [Types-sig] syntax exploration tool In-Reply-To: <15021.22659.616556.298360@anthem.wooz.org> Message-ID: I'd volunteer to put in time and help create such a tool. If someone sufficiently knowledgeable decides to go ahead with such a project please let me know. --- Nikolas Kauer > Second, if type annotations are to be seriously considered for > inclusion in Python, I think we as a community need considerable > experience with a working implementation. Yes, we need PEPs and specs > and such, but we need something real and complete that we can play > with, /without/ having to commit to its acceptance in mainstream > Python. Therefore, I think it'll be very important for type > annotation proponents to figure out a way to allow people to see and > play with an implementation in an experimental way. From nas@arctrix.com Tue Mar 13 00:13:04 2001 From: nas@arctrix.com (Neil Schemenauer) Date: Mon, 12 Mar 2001 16:13:04 -0800 Subject: [Types-sig] Re: [Python-Dev] Revive the types sig? In-Reply-To: <15021.24921.998693.156809@anthem.wooz.org>; from barry@digicool.com on Mon, Mar 12, 2001 at 06:52:57PM -0500 References: <15020.9404.557943.164934@w221.z064000254.bwi-md.dsl.cnc.net> <200103120711.AAA09711@localhost.localdomain> <15021.22659.616556.298360@anthem.wooz.org> <200103122341.SAA23054@cj20424-a.reston1.va.home.com> <15021.24921.998693.156809@anthem.wooz.org> Message-ID: <20010312161304.B2976@glacier.fnational.com> On Mon, Mar 12, 2001 at 06:52:57PM -0500, Barry A. Warsaw wrote: > I thought PTL definitely included a "template" declaration keyword, a > la, def, so they must have some solution here. MEMs guys? The correct term is "hack". We do a re.sub on the text of the module. I considered building a new parsermodule with def changed to template but haven't had time yet. I think the dominate cost when importing a PTL module is due stat() calls driven by hairy Python code. Neil From jeremy@alum.mit.edu Tue Mar 13 00:04:39 2001 From: jeremy@alum.mit.edu (Jeremy Hylton) Date: Mon, 12 Mar 2001 19:04:39 -0500 (EST) Subject: [Types-sig] Re: [Python-Dev] Revive the types sig? In-Reply-To: <15021.24921.998693.156809@anthem.wooz.org> References: <15020.9404.557943.164934@w221.z064000254.bwi-md.dsl.cnc.net> <200103120711.AAA09711@localhost.localdomain> <15021.22659.616556.298360@anthem.wooz.org> <200103122341.SAA23054@cj20424-a.reston1.va.home.com> <15021.24921.998693.156809@anthem.wooz.org> Message-ID: <15021.25623.66870.97955@w221.z064000254.bwi-md.dsl.cnc.net> >>>>> "BAW" == Barry A Warsaw writes: >>>>> "GvR" == Guido van Rossum writes: GvR> I'm not sure this is viable. I believe Jeremy's compiler GvR> package actually doesn't have its own parser -- it uses the GvR> parser module (which invokes Python's standard parse) and then GvR> transmogrifies the parse tree into something more usable, but GvR> it doesn't change the syntax! Quixote can get away with this GvR> because their only change is giving a different meaning to GvR> stand-alone string literals. But for type annotations this GvR> doesn't give enough freedom, I expect. BAW> I thought PTL definitely included a "template" declaration BAW> keyword, a la, def, so they must have some solution here. MEMs BAW> guys? I believe they use something like buf.replace("template", "def"). This won't cut it for type declarations, obviously :-). Jeremy From jepler@inetnebr.com Mon Mar 12 23:38:42 2001 From: jepler@inetnebr.com (Jeff Epler) Date: Mon, 12 Mar 2001 17:38:42 -0600 Subject: [Types-sig] MobiusPython In-Reply-To: <15021.22659.616556.298360@anthem.wooz.org> References: <15020.9404.557943.164934@w221.z064000254.bwi-md.dsl.cnc.net> <200103120711.AAA09711@localhost.localdomain> <15021.22659.616556.298360@anthem.wooz.org> Message-ID: <20010312173842.A3962@potty.housenet> On Mon, Mar 12, 2001 at 06:15:15PM -0500, Barry A. Warsaw wrote: > This might mean an extensive set of patches, a la Stackless. After > seeing and talking to Neil and Andrew about PTL and Quixote, I think > there might be another way. It seems that their approach might serve > as a framework for experimental Python syntaxes with minimal overhead. > If I understand their work correctly, they have their own compiler > which is built on Jeremy's tools, and which accepts a modified Python > grammar, generating different but compatible bytecode sequences. > E.g., their syntax has a "template" keyword approximately equivalent > to "def" and they do something different with bare strings left on the > stack. See also my project, "M=F6bius python".[1] I've used a lot of existing pieces, including the SPARK toolkit, Tools/compiler, and Lib/tokenize.py. The end result is a set of Python classes and functions that implement th= e whole tokenize/parse/build AST/bytecompile process. To the extent that each component is modifable or subclassable, Python's grammar and semanti= cs can be extended. For example, new keywords and statement types can be introduced (such as Quixote's 'tmpl'), new operators can be introduced (such as |absolute value|), along with the associated semantics. (At this time, there is only a limited potential to modify the tokenizer) One big problem right now is that M=F6bius Python only implements the 1.5.2 language subset. The CVS tree on sourceforge is not up to date, but the tree on my system = is pretty complete, lacking only documentation. Unfortunately, even a small modification requires a fair amount of code (My 'absolute value' extensio= n is 91 lines plus comments, empty lines, and imports) As far as I know, all that Quixote does at the syntax level is a few regular expression tricks. M=F6bius Python is much more than this. Jeff [1] http://mobiuspython.sourceforge.net/ From paulp@ActiveState.com Tue Mar 13 01:45:51 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Mon, 12 Mar 2001 17:45:51 -0800 Subject: [Types-sig] FOLLOWUPS!!!!!!! References: Message-ID: <3AAD7BCF.4D4F69B7@ActiveState.com> Please keep follow-ups to just types-sig. I'm very sorry I cross-posted in the beginning and I apologize to everyone on multiple lists. I did direct people to follow up only to types-sig but I should have used a header....or separate posts! -- Python: Programming the way Guido indented it. - (originated with Skip Montanaro?) From paulp@ActiveState.com Tue Mar 13 01:49:38 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Mon, 12 Mar 2001 17:49:38 -0800 Subject: [Types-sig] Proposed Goals PEP References: Message-ID: <3AAD7CB2.CF5C5A3D@ActiveState.com> "Krishnaswami, Neel" wrote: > > ... > > This isn't going to help unless you have parametrized types. The > most common case of type error in my code is when I pass in a list > or dictionary with bogus element types. This is because usually a > general list or dictionary of a particular shape is being used as > a custom "type." Errors with incorrectly handling primitive types > just doesn't seem to happen to me -- even the string/sequence > thing basically never bites me. I might be willing to put parameterized sequence and mapping types into V1 but that's my absolute outer limit. Would that be good enough? > ... > > This is a totally unrealistic expectation. Adding typechecking is > going to make typed Python code slower, not faster, for a very long > time. I'll bet it could make JPython and Python.NET faster almost immediately. They are already paying for type checks and not benefiting from unboxing. > This is for two reasons. One, the number of typechecks that the > interpreter does will go up, and without an optimization pass (which > is error prone to write) they can't be removed. Two, much of the > optimization will be from improving the representation of objects, > and that would require major surgery to the C FFI. You're probably right. But wouldn't it be realistic to say that there could be a mode where the type checks are compiled out (just like in today's -O) and the performance penalty will be basically non-existent? > This should not be a 1.0 goal -at all-, except in the modest sense > that typed code shouldn't run too much more slowly than ordinary > Python code. If you want to improve performance, focus on improving > performance as a *separate* task -- stealing Scheme interpreter > implementation tricks can win a 2-5x speedup without changing > Python's semantics at all. People keep saying that but nobody ever makes it happen... > So the order should be err, doc, glue, with no mention of opt > whatsoever. I'll demote opt and glue as side benefits as opposed to actual goals. > ... > Type inference in the presence of subtyping is still generating > research papers, which should scare you a lot. Type inference has always scared me! Ever since my experiences with ML! -- Python: Programming the way Guido indented it. - (originated with Skip Montanaro?) From paulp@ActiveState.com Tue Mar 13 01:55:33 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Mon, 12 Mar 2001 17:55:33 -0800 Subject: [Types-sig] Re: [Python-Dev] Revive the types sig? References: <15020.9404.557943.164934@w221.z064000254.bwi-md.dsl.cnc.net> <200103120711.AAA09711@localhost.localdomain> <15021.22659.616556.298360@anthem.wooz.org> Message-ID: <3AAD7E15.E6A4D5DC@ActiveState.com> "Barry A. Warsaw" wrote: > > ... > > First, it seemed to me that the greatest push for static type > annotations at IPC9 was from the folks implementing Python on top of > frameworks other than C. I know from my own experiences that there is > the allure of improved performance, e.g. JPython, given type hints > available to the compiler. While perhaps a laudable goal, this > doesn't seem to be a stated top priority of Paul's. My impression is that alot of people were interested in ERR and DOC. I don't recall much discussion of OPT at all. Everyone was just enthusing about the benefits of explicit interfaces and type checks. > Second, if type annotations are to be seriously considered for > inclusion in Python, I think we as a community need considerable > experience with a working implementation. Yes, we need PEPs and specs > and such, but we need something real and complete that we can play > with, /without/ having to commit to its acceptance in mainstream > Python. Therefore, I think it'll be very important for type > annotation proponents to figure out a way to allow people to see and > play with an implementation in an experimental way. 100% agreed! > ... > The key trick is that it all hooks together with an import hook so > normal Python code doesn't need to know anything about the mechanics > of PTL compilation. Given a homepage.ptl file, they just do an > "import homepage" and this gets magically transformed into a .ptlc > file and normal Python objects. We discussed something like this at IPC9 for implementing various experimental compiler frameworks. Unfortunately I had forgotten that compiler.py depends on Python's compiler under the hood. :( > If I've got this correct, it seems like it would be a powerful tool > for playing with alternative Python syntaxes. Ideally, the same > technique would allow the types-sig folks to create a working > implementation that would require only the installation of an import > hook. This would let them build their systems with type annotation > and prove to the skeptical among us of their overwhelming benefit. I absolutely agree. We just need a technologically reasonable way to attack the problem. One way is to hide the types stuff in strings or comments. (yuck) Another way is to embed it in standard Python syntax. (yuck) A third way is to develop an experimental compiler (based on what resources?). Suggestions are welcome... -- Python: Programming the way Guido indented it. - (originated with Skip Montanaro?) From paulp@ActiveState.com Tue Mar 13 01:59:00 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Mon, 12 Mar 2001 17:59:00 -0800 Subject: [Types-sig] MobiusPython References: <15020.9404.557943.164934@w221.z064000254.bwi-md.dsl.cnc.net> <200103120711.AAA09711@localhost.localdomain> <15021.22659.616556.298360@anthem.wooz.org> <20010312173842.A3962@potty.housenet> Message-ID: <3AAD7EE4.E6AFA6D9@ActiveState.com> Jeff Epler wrote: > > > ... > > I've used a lot of existing pieces, including the SPARK toolkit, I'm worried about the slowness of the SPARK toolkit. I've seen exponential behavior with it before and Eric Raymond just mentioned some on python-dev. You really have to know what you are doing to get good performance and I don't! Hasn't anyone implemented a Python compiler in mxtexttools or yapps or any of the other parser generator systems? -- Python: Programming the way Guido indented it. - (originated with Skip Montanaro?) From ping@lfw.org Tue Mar 13 01:56:27 2001 From: ping@lfw.org (Ka-Ping Yee) Date: Mon, 12 Mar 2001 17:56:27 -0800 (PST) Subject: [Types-sig] Re: [Python-Dev] parsers and import hooks In-Reply-To: <20010312160729.A2976@glacier.fnational.com> Message-ID: On Mon, 12 Mar 2001, Neil Schemenauer wrote: > > Its nice if you can get it to work. import hooks are a bitch to > write and are slow. Also, you get trackbacks from hell. It > would be nice if there were higher level hooks in the > interpreter. Let me chime in with a request, please, for a higher-level find_module() that understands packages -- or is there already some way to emulate the file-finding behaviour of "import x.y.z" that i don't know about? -- ?!ng "Computers are useless. They can only give you answers." -- Pablo Picasso From paulp@ActiveState.com Tue Mar 13 02:00:46 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Mon, 12 Mar 2001 18:00:46 -0800 Subject: [Types-sig] Re: [Python-Dev] Revive the types sig? References: <15020.9404.557943.164934@w221.z064000254.bwi-md.dsl.cnc.net> <200103120711.AAA09711@localhost.localdomain> <15021.22659.616556.298360@anthem.wooz.org> <200103122341.SAA23054@cj20424-a.reston1.va.home.com> <15021.24398.737816.108344@w221.z064000254.bwi-md.dsl.cnc.net> Message-ID: <3AAD7F4E.6B0A460F@ActiveState.com> Jeremy Hylton wrote: > >... > > Right! It would be nice to kill two birds with one stone though. We > could generate an alternate parser for Python and have it generate the > same AST, decorated with type information. Even if the type > experiment doesn't work out, the alternate parser might be useful. I would love to see this for a variety of reasons. I have bandwidth to generate types-sig patches for a compiler but not to write the compiler from scratch, though. :( Anyone else interested? -- Python: Programming the way Guido indented it. - (originated with Skip Montanaro?) From barry@digicool.com Tue Mar 13 02:56:42 2001 From: barry@digicool.com (Barry A. Warsaw) Date: Mon, 12 Mar 2001 21:56:42 -0500 Subject: [Types-sig] Re: [Python-Dev] Revive the types sig? References: <15020.9404.557943.164934@w221.z064000254.bwi-md.dsl.cnc.net> <200103120711.AAA09711@localhost.localdomain> <15021.22659.616556.298360@anthem.wooz.org> <200103122341.SAA23054@cj20424-a.reston1.va.home.com> <15021.24921.998693.156809@anthem.wooz.org> <20010312161304.B2976@glacier.fnational.com> Message-ID: <15021.35946.606279.267593@anthem.wooz.org> >>>>> "NS" == Neil Schemenauer writes: >> I thought PTL definitely included a "template" declaration >> keyword, a la, def, so they must have some solution here. MEMs >> guys? NS> The correct term is "hack". We do a re.sub on the text of the NS> module. I considered building a new parsermodule with def NS> changed to template but haven't had time yet. I think the NS> dominate cost when importing a PTL module is due stat() calls NS> driven by hairy Python code. Ah, good to know, thanks. I definitely think it would be A Cool Thing if one could build a complete Python parser and compiler in Python. Kind of along the lines of building the interpreter main loop in Python as much as possible. I know that /I'm/ not going to have any time to contribute though (and others have more and better experience in this area than I do). -Barry From paulp@ActiveState.com Tue Mar 13 03:09:21 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Mon, 12 Mar 2001 19:09:21 -0800 Subject: [Types-sig] Re: [Python-Dev] Revive the types sig? References: <15020.9404.557943.164934@w221.z064000254.bwi-md.dsl.cnc.net> <200103120711.AAA09711@localhost.localdomain> <15021.22659.616556.298360@anthem.wooz.org> <200103122341.SAA23054@cj20424-a.reston1.va.home.com> <15021.24921.998693.156809@anthem.wooz.org> <20010312161304.B2976@glacier.fnational.com> <15021.35946.606279.267593@anthem.wooz.org> Message-ID: <3AAD8F61.C61CAC85@ActiveState.com> "Barry A. Warsaw" wrote: > >... > > Ah, good to know, thanks. I definitely think it would be A Cool Thing > if one could build a complete Python parser and compiler in Python. > Kind of along the lines of building the interpreter main loop in > Python as much as possible. I know that /I'm/ not going to have any > time to contribute though (and others have more and better experience > in this area than I do). I'm surprised that there are dozens of compiler compilers written in Python but few people stepped forward to say that theirs supports Python itself. mxTextTools has a Python parser...does anyone know how good it is? -- Python: Programming the way Guido indented it. - (originated with Skip Montanaro?) From jepler@inetnebr.com Tue Mar 13 04:11:24 2001 From: jepler@inetnebr.com (Jeff Epler) Date: Mon, 12 Mar 2001 22:11:24 -0600 Subject: [Types-sig] MobiusPython In-Reply-To: <3AAD7EE4.E6AFA6D9@ActiveState.com> References: <15020.9404.557943.164934@w221.z064000254.bwi-md.dsl.cnc.net> <200103120711.AAA09711@localhost.localdomain> <15021.22659.616556.298360@anthem.wooz.org> <20010312173842.A3962@potty.housenet> <3AAD7EE4.E6AFA6D9@ActiveState.com> Message-ID: <20010312221124.A4640@potty.housenet> On Mon, Mar 12, 2001 at 05:59:00PM -0800, Paul Prescod wrote: > I'm worried about the slowness of the SPARK toolkit. I've seen > exponential behavior with it before and Eric Raymond just mentioned some > on python-dev. You really have to know what you are doing to get good > performance and I don't! I saw Eric's message. What a coincidence that it comes at the same time as mine. I don't understand the maths behind the time complexities of the "Earley" parser either (I stop just after 'bubble sort is O(n^2), and that's bad'), but I'll defer to John Aycock, who says that The time complexity of Earley's algorithm is O(n^3) in the worst case, that being the meanest, nastiest, most ambiguous context-free grammar you could possibly think of. Unless you're parsing natural language, this won't happen. For any unambiguous grammar, the worst case drops to O(n^2), and for a set of grammars which loosely coincides with the LR(k) grammars, the complexity drops to O(n). I suspect that the Python grammar is in the last group, and thus has the desirable O(n) complexity. In any case, the fact that Mobius is based on SPARK shouldn't make Mobius unsuitable for this task even if SPARK is slow, if the intention is *prototyping* the extensions. Once the syntax is solid, it can be coded directly into Python-2.x/Grammar/Grammar. (That said, the performance of my parser is not great, to the extent that it takes about 1 second/LOC on my 486-75 laptop, where I often develop Mobius --- so far, speed has not been my goal, passing the parser test suite is) Jeff From paulp@ActiveState.com Tue Mar 13 06:58:46 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Mon, 12 Mar 2001 22:58:46 -0800 Subject: [Types-sig] Re: Revive the types sig? References: <3AACFD18.E4CE0115@ActiveState.com> Message-ID: <3AADC526.59DE2468@ActiveState.com> Which of the ideas in the paper do you feel are applicable to a system with explicitly declared types? Thanks for pointers! -- Python: Programming the way Guido indented it. - (originated with Skip Montanaro?) From tim.one@home.com Tue Mar 13 07:41:37 2001 From: tim.one@home.com (Tim Peters) Date: Tue, 13 Mar 2001 02:41:37 -0500 Subject: [Types-sig] Proposed Goals PEP In-Reply-To: <175b01c0ab1a$ba638e80$87740918@CX781526B> Message-ID: [Tim Hochberg, perhaps the only person so far in this round to actually talk about how to spell and implement a workable type annotation system for Python ... ] Good show, fellow Tim! > THE BASICS > > Consider this code (syntax borrowed from one of Guido's old proposals): > > def spam(eggs : Integer, beans : File) -> String: > #.... > return x > > This would be compiled to: > > def spam(eggs, bakedBeans): > if __typecheck__ and not Integer.check(eggs): raise > TypeError("Informative message here") > if __typecheck__ and not File.check(beans): raise TypeError("Informative > message here") > # ... > if __typecheck__ and not String.check(x): raise TypeError("Informative > message here") > return x > > Note that Integer, File and String are all bound when the def is executed, > just like default argument are now. Also, __typecheck__ is magic variable > similar to debug so that typechecks can be omitted completely during > compilation so that there is no speed penalty when compiled this way. Everybody got that? The rest (see Tim's msg) is elaboration. Type checkers are spelled via instances of classes with .check(object) methods. Everything else follows from that. You can get all the ERR and DOC benefits anyone could ask for within a week; and wrt ERR, anything can be checked that you can write Python code *to* check. OPT and GLUE aren't particularly helped at all. Tough. I suggest changing the codegen like so: def f(p1: T1, p2: T2, ..., pn: Tn) -> Tr: if __typecheck__: T1.check(p1, msg1) T2.check(p2, msg2) ... Tn.check(pn, msgn) 1. It's less code. 2. The compiler today isn't actually smart enough to optimize away if 0 and y: xxx blocks. It is smart enough to optimize away if 0: xxx blocks (same deal with __debug__ today). 3. The .check() method is the proper place to raise an exception. 4. But the calling code is the proper place to construct part of the error message, because it has easy access to useful symbolic info about the function (like the names of the arguments). return expr needs to change into (in general) _unique_temp = expr if __typecheck__: Tr.check(_unique_temp, msgr) return _unique_temp which is less trivial to implement via a source-to-source translator, but still easy. Before: def lastelement(t: AnyType, x: SequenceOf(t)) -> t: return x[-1] After: if __typecheck__: # typecheck is a module containing std check objects like # AnyType and SequenceOf, etc. # Of course users can code their own check objects too. from typecheck import * def lastelement(t, x): if __typecheck__: AnyType.check(t, "lastelement argument t") SequenceOf(t).check(x, "lastelement argument x") temp = x[-1] if __typecheck__: t.check(temp, "lastelement return expression x[-1]") return temp Subtlety: the "t" argument above has to be a check object itself. That means AnyType.check(t) has to be able to determine whether t *is* a check object. This would be a great time to insist that all check objects be instances of subclasses of a base TypeCheck class, so that isinstance(t, TypeCheck) is sufficient. I don't see that anything would be gained by continuing the usual dance of saying "well, if it has a .check() attribute, then I can't prove it's *not* a typecheck object, so I'll just assume that it is". > ... > SUMMARY > > This idea looks like a good starting point to me. It's simple, it can > express complex types, it helps both err and doc. On the downside, it's > unlikely to help opt, and I don't know about glue. Take heart: the same people who believe Python can magically infer types can't deny that it could just as magically track down the defns of the check objects and infer everything useful about them too . if-the-implementation-is-easy-to-explain-it-may-be-a-good-idea-ly y'rs - tim From paulp@ActiveState.com Tue Mar 13 11:49:38 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Tue, 13 Mar 2001 03:49:38 -0800 Subject: [Types-sig] Proposed Goals PEP References: <3AACF746.9A06058F@ActiveState.com> <175b01c0ab1a$ba638e80$87740918@CX781526B> Message-ID: <3AAE0952.CF9D58DA@ActiveState.com> I'll consider your idea. I've seen it before (under various syntaxes and names) and I can certainly see it's strengths. I have to think carefully before I throw away any hope of ever statically typing these things so decisively. Once you depend on .check methods (called "implementedBy" in the scarecrow interfaces model) you have made static type checking equivalent to the halting problem! It is one thing to not go far out of your way to help static type checkers (which is what I tried to do in the past) and another to actively get in their way! Still, I'm thinking about it...it does have a certain simplicity. I think that it will probably be a basis at least for experimentation and maybe for the final PEP. -- Python: Programming the way Guido indented it. - (originated with Skip Montanaro?) From paulp@ActiveState.com Tue Mar 13 12:10:50 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Tue, 13 Mar 2001 04:10:50 -0800 Subject: [Types-sig] Proposed Goals PEP References: Message-ID: <3AAE0E4A.D7E7A58E@ActiveState.com> Tim Peters wrote: > > ... > > 3. The .check() method is the proper place to raise an exception. I disagree. foo.check() should be something that you could call anywhere in Python code and it should return a boolean. It should also probably be called __check__ or __typecheck__ or __implementedby__ or something similar. It is invoked magically under the covers like all of the other __ methods. if String.check(a): dosomething else: dosomethingelse > 4. But the calling code is the proper place to construct part of the > error message, because it has easy access to useful symbolic info > about the function (like the names of the arguments). Another good reason to have check return a boolean. > return expr > > needs to change into (in general) > > _unique_temp = expr > if __typecheck__: > Tr.check(_unique_temp, msgr) > return _unique_temp > > which is less trivial to implement via a source-to-source translator, but > still easy. I was thinking of implementing it as a byte-code transformation which might be easier. 0 SET_LINENO 1 3 SET_LINENO 2 6 LOAD_FAST 0 (a) 9 PRINT_ITEM 10 PRINT_NEWLINE 11 SET_LINENO 3 14 LOAD_FAST 1 (b) 17 PRINT_ITEM 18 PRINT_NEWLINE 19 SET_LINENO 4 22 LOAD_FAST 1 (b) 25 LOAD_FAST 0 (a) 28 BUILD_TUPLE 2 (typecheck goes HERE!) 31 RETURN_VALUE 32 LOAD_CONST 0 (None) 35 RETURN_VALUE >... > > Subtlety: the "t" argument above has to be a check object itself. That > means AnyType.check(t) has to be able to determine whether t *is* a check > object. This would be a great time to insist that all check objects be > instances of subclasses of a base TypeCheck class, so that isinstance(t, > TypeCheck) is sufficient. I don't see that anything would be gained by > continuing the usual dance of saying "well, if it has a .check() attribute, > then I can't prove it's *not* a typecheck object, so I'll just assume that it > is". The reason to do the dance is to allow a C-type to implement .check. The middle ground is to require an __implements__ that contains TypeCheck (or a subtype). > > SUMMARY > > > > This idea looks like a good starting point to me. It's simple, it can > > express complex types, it helps both err and doc. On the downside, it's > > unlikely to help opt, and I don't know about glue. > > Take heart: the same people who believe Python can magically infer types > can't deny that it could just as magically track down the defns of the check > objects and infer everything useful about them too . Those are the wrong people to worry about. They say any type declarations as cruft anyhow. But there are some people who want to use this information in static ways. I had hoped not to get in their way! On the other hand, the thing that the speed-demons REALLY need is a way to freeze namespaces -- instance, class and module namespaces. If they "freeze" the types module namespace then they can be assured that the Integer type is the REAL integer type and they can optimize integer inputs as real integers. Furthermore, there is a natural upgrade path from this model to the scarecrow interfaces model. That's more of a static-y type of thing for those who want that. -- Python: Programming the way Guido indented it. - (originated with Skip Montanaro?) From guido@digicool.com Tue Mar 13 12:06:21 2001 From: guido@digicool.com (Guido van Rossum) Date: Tue, 13 Mar 2001 07:06:21 -0500 Subject: [Types-sig] Proposed Goals PEP In-Reply-To: Your message of "Tue, 13 Mar 2001 03:49:38 PST." <3AAE0952.CF9D58DA@ActiveState.com> References: <3AACF746.9A06058F@ActiveState.com> <175b01c0ab1a$ba638e80$87740918@CX781526B> <3AAE0952.CF9D58DA@ActiveState.com> Message-ID: <200103131206.HAA30768@cj20424-a.reston1.va.home.com> > I'll consider your idea. I've seen it before (under various syntaxes and > names) and I can certainly see it's strengths. I have to think carefully > before I throw away any hope of ever statically typing these things so > decisively. Once you depend on .check methods (called "implementedBy" in > the scarecrow interfaces model) you have made static type checking > equivalent to the halting problem! It is one thing to not go far out of > your way to help static type checkers (which is what I tried to do in > the past) and another to actively get in their way! > > Still, I'm thinking about it...it does have a certain simplicity. I > think that it will probably be a basis at least for experimentation and > maybe for the final PEP. Note that he was also proposing a syntax that does not necessarily depend on .check() methods! --Guido van Rossum (home page: http://www.python.org/~guido/) From paulp@ActiveState.com Tue Mar 13 12:42:16 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Tue, 13 Mar 2001 04:42:16 -0800 Subject: [Types-sig] Framework for an experimental type system Message-ID: <3AAE15A8.640E223@ActiveState.com> The fastest way to get to a working type system is to get to an experimental type system and play with it. The fastest way to get to an experimental type system is to stay as close as possible to Python's existing syntax. So I'm going to use Tim's idea as elaborated by Tim. Thanks Tim and Tim! And for those who are not named Tim and would like credit, I'll point out that this is not the first time we've seen this idea. Now that we've shifted goals it seems more workable than it did when we were very concerned about optimization. Here is a proposed experimental strategy: from __experimental__.typecheck import * def open(filename, mode="r", buffering=-1): assertTypes(filename=String, mode=String, buffering=Integer, _RC=Integer}) ... This model allows us to build the type-system right this minute. At first, it only helps err, not doc or anything else but that's okay. It's a basis for experimentation. _RC is a magic constant which declares the return type of this method. If we *require* the assertTypes function to be the first thing in the method after the (optional) docstring then we can find it predictably and use it for doc, just as if the declarations were right in the open function. The types come from "typecheck" and they all implement an __implementedby__ method. Also, it should be possible to assert conformance to a type with an __interfaces__ attribute. In other words, Tim's Type objects and Michel's Interface objects are the same thing. Michel happens to have a syntactic trick that is very nice for declaring OO interfaces but it was always the understanding that it comes down to a runtime check. If a type declaration points to a class (not an object), then we should use instanceof under the covers, not class.__implementedby__. class Person: ... def hire(employee): assertTypes(employee=Person) One downside about Tim's model is that a Type object that inherits from multiple other type objects will have to manually delegate to each of their __implementedby__ methods (barring __getattr__ tricks). I think that maybe the typeAssert function should walk up the inheritance tree testing ALL __implementedby__ methods on base classes. -- Python: Programming the way Guido indented it. - (originated with Skip Montanaro?) From pedroni@inf.ethz.ch Tue Mar 13 13:27:04 2001 From: pedroni@inf.ethz.ch (Samuele Pedroni) Date: Tue, 13 Mar 2001 14:27:04 +0100 Subject: [Types-sig] opt need a bit of static behaviour (was: Proposed Goals PEP) References: <3AAE0E4A.D7E7A58E@ActiveState.com> Message-ID: <002f01c0abc1$4cecd080$705821c0@newmexico> Hi. > On the other hand, the thing that the speed-demons REALLY need is a way > to freeze namespaces -- instance, class and module namespaces. If they > "freeze" the types module namespace then they can be assured that the > Integer type is the REAL integer type and they can optimize integer > inputs as real integers. Yes, it is the old problem that you can optimize e.g. range only if you're sure range is what is supposed to be. regards. From paulp@ActiveState.com Tue Mar 13 13:35:06 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Tue, 13 Mar 2001 05:35:06 -0800 Subject: [Types-sig] Sample declarations Message-ID: <3AAE220A.DC92ABED@ActiveState.com> Here are some sample declarations from an experimental typecheck module. I'd like feedback because there are lots of subtle issues. import types, operator ############# Basic types ############## class _number: "Any kind of Number" def __implementedby__(self, obj): try: # includes appropriate complexes, excludes strings # should I use isNumberType??? complex(obj) return 1 except TypeError: return 0 Number=_number() Complex=_number() Complex.__doc__="A complex number (including one with no imaginary part)" class _integer: "An integer value" def __implementedby__(self, obj): try: # includes appropriate floats and complexes return int(obj)==obj except TypeError: return 0 Integer=_integer() class _float: "A floating point value" def __implementedby__(self, obj): try: # includes integers, appropriate complexes return float(obj)==obj except TypeError: return 0 Float=_float() class _string: "Any kind of string" def __implementedby__(self, obj): return obj in (types.StringType, types.UnicodeType) String=_string() ########### Ancient Protocols ################# def findInterface(iface, obj): ifaces = getattr(obj, "__interfaces__", None) # don't make this mutually recursive on # Sequence.__implementedby__! if type(ifaces) in (types.ListType, types.TupleType): return iface in ifaces # multiple interfaces case else: return iface is ifaces # single interface case class _sequence: "Any kind of sequence" def __implementedby__(self, obj): if findInterface(self, obj): return 1 else: return operator.isSequenceType(obj) Sequence=_sequence() class _mapping: "Any kind of mapping from keys to values" def __implementedby__(self, obj): if findInterface(self, obj): return 1 else: return operator.isMappingType(obj) Mapping=_sequence() class _callable: "Any object that can be called with arguments." def __implementedby__(self, obj): return callable(obj) Callable=_callable() class _readstream: "An object that can be read from like a file." def __implementedby__(self, obj): if findInterface(self, obj): return 1 else: attrs = dir(obj) #argh...this doesn't handle instances well! return ("read" in attrs and "flush" in attrs and "seek" in attrs) # ... ReadStream= _readstream() def experimentalAssertTypes(dict): # this is VERY ROUGH # for one thing it should use variable names from the caller, # not a passed in dictionary! for val, type in dict.items(): if not type.__implementedby__(val): raise TypeError(val, type) def _test(): import os # use doctest eventually! experimentalAssertTypes({5:Integer}) experimentalAssertTypes({_test:Callable}) experimentalAssertTypes({os.popen("blah"):ReadStream}) # experimentalAssertTypes({5:String}) # should raise assertion if __name__=="__main__": _test() -- Python: Programming the way Guido indented it. - (originated with Skip Montanaro?) From guido@digicool.com Tue Mar 13 14:52:10 2001 From: guido@digicool.com (Guido van Rossum) Date: Tue, 13 Mar 2001 09:52:10 -0500 Subject: [Types-sig] Sample declarations In-Reply-To: Your message of "Tue, 13 Mar 2001 05:35:06 PST." <3AAE220A.DC92ABED@ActiveState.com> References: <3AAE220A.DC92ABED@ActiveState.com> Message-ID: <200103131452.JAA32144@cj20424-a.reston1.va.home.com> > Here are some sample declarations from an experimental typecheck module. > I'd like feedback because there are lots of subtle issues. I think you're on a slippery slope with some assumptions about numeric types here: - Your test for number assumes that something can be converted to complex. That may not be true. - Your test for Integer includes float/complex numbers with integral values; that doesn't work, e.g. a[1.0] raises TypeError if a is a sequence. - Your test for Float values that accepts ints is also bad news, at least until integer division is fixed. You may want to define a separate type IntOrFloat, for those cases where the programmer asserts that either type is fine, and another IntOrFloatOrComplex. > def findInterface(iface, obj): > ifaces = getattr(obj, "__interfaces__", None) > # don't make this mutually recursive on > # Sequence.__implementedby__! > if type(ifaces) in (types.ListType, types.TupleType): > return iface in ifaces # multiple interfaces case > else: > return iface is ifaces # single interface case This helper function is steeped in irrelevant implementation detail! - Your assumptions about sequences, mappings etc. are also pretty naive. operator.isSequenceType() and .isMappingType() return true for all class instances regardless of what they actually implement, and should this be avoided. - One way to go would be to have explicit ways to say "x is any Sequence", "x is a List", "x is any Mapping", "x is a Dictionary". - But note again that Sequence-ness and Mapping-ness are ill-defined. Look at this: def C: def __getitem__(self, x): return x x = C() Is x a sequence? A mapping? It could fake being either! Before you can meaningfully discuss the Sequence check code, you should make up your mind when you start calling something a sequence. Clearly it needs __getitem__ (let's limit ourselves to class instances). Does it need __len__? Does it need __getslice__? Does it need to have count() and index()? reverse() and sort()? You also need a separate type to indicate a writable sequence. For mappings, there's a whole slew of methods that are informally part of the mapping protocol yet aren't implemented by all mappings, e.g. has_key(), keys(), values(), items(), get(), copy(), and for mutable mappings, clear(), setdefault(), update(). - Similar questions can be asked about streams. In short, it looks (and I am as surprised as you are!!!) like we will need to agree on the Python type hierarchy before we can ever hope to agree on any kind of type assertions. That is, unless you make the type assertions only deal with the concrete types. But that wouldn't be very useful, as there are lots of cases where you want to express that you accept any kind of sequence, mapping, file, etc. --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.hochberg@ieee.org Tue Mar 13 15:03:44 2001 From: tim.hochberg@ieee.org (Tim Hochberg) Date: Tue, 13 Mar 2001 08:03:44 -0700 Subject: [Types-sig] Sample declarations References: <3AAE220A.DC92ABED@ActiveState.com> Message-ID: <190801c0abce$cc5aa880$87740918@CX781526B> Paul Prescod writes: ----- Original Message ----- > ############# Basic types ############## > > class _number: > "Any kind of Number" > def __implementedby__(self, obj): > try: # includes appropriate complexes, excludes strings > # should I use isNumberType??? > complex(obj) > return 1 > except TypeError: > return 0 This probably isn't what you want since it returns true for strings that represent complex numbers, such as "1+5j". Perhaps you should use your complex(obj) == obj trick here as you did for float. On the other hand, that has the same problems with integers that Guido notes in his email. I see Guido has now commented on the others ones more thouroughly than I could have, so I'll leave it at that. -tim (who went to bed thinking this proposal had vanished without ripple...) From Samuele Pedroni Tue Mar 13 15:14:28 2001 From: Samuele Pedroni (Samuele Pedroni) Date: Tue, 13 Mar 2001 16:14:28 +0100 (MET) Subject: [Types-sig] Sample declarations Message-ID: <200103131514.QAA17208@core.inf.ethz.ch> Hi. [GvR] > > - But note again that Sequence-ness and Mapping-ness are ill-defined. > Look at this: > > def C: > def __getitem__(self, x): return x > > x = C() > > Is x a sequence? A mapping? It could fake being either! Before > you can meaningfully discuss the Sequence check code, you should > make up your mind when you start calling something a sequence. > Clearly it needs __getitem__ (let's limit ourselves to class > instances). Does it need __len__? Does it need __getslice__? Does > it need to have count() and index()? reverse() and sort()? You > also need a separate type to indicate a writable sequence. For > mappings, there's a whole slew of methods that are informally part > of the mapping protocol yet aren't implemented by all mappings, > e.g. has_key(), keys(), values(), items(), get(), copy(), and for > mutable mappings, clear(), setdefault(), update(). > > - Similar questions can be asked about streams. > > In short, it looks (and I am as surprised as you are!!!) like we will > need to agree on the Python type hierarchy before we can ever hope to > agree on any kind of type assertions. That is, unless you make the > type assertions only deal with the concrete types. But that wouldn't > be very useful, as there are lots of cases where you want to express > that you accept any kind of sequence, mapping, file, etc. (My external viewpoint) The point is that until now python let one be lazy and free. Clearly we can introduce a hierarchy of so to say abstract classes, that at least mention all the methods that a class should implement to be a Mapping, a Sequence, ... But my feeling is that at the moment python is more like Self or Smalltalk an input is fine for an algorithm as long it defines just the required "selectors" (methods) in a meaningful way. Is that a bad practice wrt to err&doc? probably yes but allows for flexibility and quick coding. If for example the whole std lib will introduce typechecking that could be a problem. regards, Samuele Pedroni. From othello@javanet.com Tue Mar 13 16:31:04 2001 From: othello@javanet.com (Raymond Hettinger) Date: Tue, 13 Mar 2001 11:31:04 -0500 Subject: [Types-sig] Ready for the next step Message-ID: <3AAE4B48.A5F05A9E@javanet.com> We've got two wonderful suggestions on the table: Ka-Ping Yee's class Spam(Interface) Tim Hochberg's def spam(egg:Integer, beans:File) The two suggestions do not conflict, meet many of the project goals, and are straightforward to implement. Let's get wild, bundle the two together in PEP and see if we can all get behind it. been-known-to-jump-the-gun-ly yours, Raymond "Type-sigs is Alive, oh yah." From danwang@CS.Princeton.EDU Tue Mar 13 16:46:00 2001 From: danwang@CS.Princeton.EDU (Daniel Wang) Date: 13 Mar 2001 11:46:00 -0500 Subject: [Types-sig] Re: Revive the types sig? In-Reply-To: Paul Prescod's message of "Mon, 12 Mar 2001 22:58:46 -0800" References: <3AACFD18.E4CE0115@ActiveState.com> <3AADC526.59DE2468@ActiveState.com> Message-ID: Paul Prescod writes: > Which of the ideas in the paper do you feel are applicable to a system > with explicitly declared types? > > Thanks for pointers! > Burried behind all that math is a recipe for deciding where to put dynamic type checks and tagging functions that move your data safely between the typed and untyped world. It properly handles parameterized types. The short list in the intro promises a system that will accept any Scheme program without modification but run it like an ML program when you use types in an ML like way. Sounds like what the Python folk would like in ideal world. You might not be interested in doing full inference in which case you can just "reverse" engineer the system to see where they need to put casts and tagging operations to make things work. These are exactly the same places a Python system would have to require programmer annotation. Anyway, it's been a while since I read it, but its the most relevant bit of acadmeic type-theory I know about for this partricular problem. From paulp@ActiveState.com Tue Mar 13 17:15:45 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Tue, 13 Mar 2001 09:15:45 -0800 Subject: [Types-sig] Ready for the next step References: <3AAE4B48.A5F05A9E@javanet.com> Message-ID: <3AAE55C1.3CEF8E07@ActiveState.com> Raymond Hettinger wrote: > > We've got two wonderful suggestions on the table: > > Ka-Ping Yee's class Spam(Interface) Ka-Ping's proposal overlaps alot with the work they've been doing at Digital Creations for the last, oh, 4 years. I think we either need a PEP from each or they have to get together and unify their proposals. > Tim Hochberg's def spam(egg:Integer, beans:File) > > The two suggestions do not conflict, meet many of the > project goals, and are straightforward to implement. Yeah, it's straightforward to implement until you try to implement it! Python doesn't have a proper type/interface hierarchy. We still need to figure that. -- Python: Programming the way Guido indented it. - (originated with Skip Montanaro?) From qrczak@knm.org.pl Tue Mar 13 17:16:46 2001 From: qrczak@knm.org.pl (Marcin 'Qrczak' Kowalczyk) Date: 13 Mar 2001 17:16:46 GMT Subject: [Types-sig] Proposed Goals PEP References: <3AACF746.9A06058F@ActiveState.com> <175b01c0ab1a$ba638e80$87740918@CX781526B> Message-ID: Mon, 12 Mar 2001 10:34:44 -0700, Tim Hochberg pis= ze: > Parameterized types are easy to support in this model, although > the checks are potentially very expensive. When a function returns a list of ints and happens to return an empty list, and another function expects a list of tuples and gets this empty list, IMHO it should be an error. This, and the fact that we don't want to iterate over the list each time it is passed through a typeful gate, suggests that a list should store the type of its elements in its attribute. When a list is typechecked for the first time, items are checked and the type is remembered. From this time the invariant is that it contains only items of the right type. Further typechecking won't iterate over elements but look at the stored type. Typechecking of future items will happen at the time of their insertion. Similarly, the type of a function cannot be derived from the current function object (you can't apply the function just to see what type it will return) - it must be stored in it explicitly. > Consider a parameterized list type: I would certainly like to use real type objects and class objects in type expressions. Unfortunately it creates a syntactic ambiguity when user-defined parametrized types are involved: ClassName(Type1, Type2) This looks like a constructor call. We don't want this kind of overloading. It can be worked around by using a different syntax for type application, e.g. ClassName[Type1, Type2] For consistency this extends to builtin types, so we should have List[String] Tuple[Int, Int, Int] Dict[String, Tuple[Int, Int]] If and only if type expressions were evaluated under different rules, we could have a nicer syntax: [String] (Int, Int, Int) {String: (Int, Int)} But I don't propose this now. You can ignore type parameters if you wish. Bare Dict means Dict[Any,Any] (because it is implemented that way). You can also apply a class object to types and use it as a class. E.g. UserList() makes a UserList which is typeless yet, but you can also write UserList[Int]() which will accept only Ints. Usually you will just write UserList() and wait until it is passed somewhere where the type is written down, e.g. returned from a function with the return type of Sequence[String]. The UserList becomes a UserList of strings then. Applying a type constraint to a mutable object may change the stored type of its elements. > class CompositeOf: > "A composite type" > def __init__(self, *types): > self.types =3D types > self.name =3D "CompositeOf(%s)" % ", ".join([t.name for t in type= s]) > def check(self, object): > for t in types: > if types.check(t): return 1 > return 0 You mean a sum type. It can be done using the | operator. When applied to two types, it produces the right composite type. Similarly &, which is really type unification. & will not build a composite type which checks that a value belongs to both types, but unify types right away, raising an exception when it's not possible. Typechecking uses unification internally. A list is not in a separate typeless state initially - it remembers its element type as Any. Added elements are typechecked wrt. Any, which is trivial (always satisfied). When a type constraint is applied to a list, its element type is unified with that of the constraint. Usually it happens once when the type is changed from Any to some concrete type (Any & t =3D t), and many times later when the concrete type is confirmed (t & t =3D t). The list must typecheck all its elements only when the declared type changes. So types belong to objects, not to names. You can rebind a local variable no matter how much its old and new types disagree. But collections may impose restrictions on types of their elements and adding an element to a collection is not always type correct. These restrictions result from explicitly written type constraints over the collection. There will be a scheme of declaring types of instance attributes. The instance will then make sure that a newly assigned object has a correct type. --=20 __("< Marcin Kowalczyk * qrczak@knm.org.pl http://qrczak.ids.net.pl/ \__/ ^^ SYGNATURA ZAST=CAPCZA QRCZAK From paulp@ActiveState.com Tue Mar 13 17:18:12 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Tue, 13 Mar 2001 09:18:12 -0800 Subject: [Types-sig] Sample declarations References: <3AAE220A.DC92ABED@ActiveState.com> <200103131452.JAA32144@cj20424-a.reston1.va.home.com> Message-ID: <3AAE5654.60539291@ActiveState.com> Guido van Rossum wrote: > >... > > I think you're on a slippery slope with some assumptions about numeric > types here: > > - Your test for number assumes that something can be converted to > complex. That may not be true. What's an appropriate definition for number? operator.isNumberType()? > - Your test for Integer includes float/complex numbers with integral > values; that doesn't work, e.g. a[1.0] raises TypeError if a is a > sequence. And yet this works: open("c:\\autoexec.bat", "r", 5.0).read() Is it by design that these two things work differently? Or is it an implementation quirk? And if it is an implementation quirk then maybe we should use this as an opportunity to make the language more consistent by loosening up some type checks. My impression is that PyArg_ParseTuples is pretty flexible and it made sense for the Python equivalent of PyArg_ParseTuples to be as similar as possible. Of course one big definition between PyArg_ParseTuples and __implementedby__ is that the former actually coerces its args and the latter only checks them. Maybe this is a design flaw.... If there is an obvious way to coerce an object to a type, we should probably use it. This will make the whole system's behavior more predictable. But this doesn't work: coerce(5, types.FloatType) In general I don't know how to try to coerce objects to types. I only know how to coerce two objects to some common type. > - Your test for Float values that accepts ints is also bad news, at > least until integer division is fixed. You may want to define a > separate type IntOrFloat, for those cases where the programmer > asserts that either type is fine, and another IntOrFloatOrComplex. Not entirely happy with that design!!! I'd rather change the language (which we already intend to fix) rather than hack up the type system. > > def findInterface(iface, obj): > > ifaces = getattr(obj, "__interfaces__", None) > > # don't make this mutually recursive on > > # Sequence.__implementedby__! > > if type(ifaces) in (types.ListType, types.TupleType): > > return iface in ifaces # multiple interfaces case > > else: > > return iface is ifaces # single interface case > > This helper function is steeped in irrelevant implementation detail! I'm implementing something! I have to put in all of the details required to make it work. It's supposed to be a prototype not pseudocode. > - Your assumptions about sequences, mappings etc. are also pretty > naive. operator.isSequenceType() and .isMappingType() return true > for all class instances regardless of what they actually implement, > and should this be avoided. I had an explicit check for InstanceType-ness but accidentally deleted it. I thought I had incorporated it into findInterface. I'll put it back in. Here's Sequence, fixed up: class _sequence: "Any kind of sequence" def __implementedby__(self, obj): if findInterface(self, obj): # if an object claims to support return 1 # the protocol, we trust it! elif type(obj)==types.InstanceType: # no safe way to know if an instance return 0 # supports the protocol else: return operator.isSequenceType(obj) # safe for C-types Sequence=_sequence() > - One way to go would be to have explicit ways to say "x is any > Sequence", "x is a List", "x is any Mapping", "x is a Dictionary". We do: def foo(x: Sequence, x:ListType, x:Mapping, x:DictionaryType): ... I haven't implemented the part that checks type objects (or class objects, for that matter). Here's a new version with an implementation for that: def experimentalAssertTypes(dict): # this is VERY ROUGH # for one thing it should use variable names from the caller, # not a passed in dictionary! for val, typ in dict.items(): if type(typ) in (types.ClassType, types.TypeType): if not isinstance(val, typ): raise TypeError(val, typ) elif typ.__implementedby__(val): continue else: raise TypeError(val, typ) > - But note again that Sequence-ness and Mapping-ness are ill-defined. I agree. This comes back to the issue of interfaces. I think that the DC guys have made some progress on a standard type hierarchy, haven't they? > In short, it looks (and I am as surprised as you are!!!) like we will > need to agree on the Python type hierarchy before we can ever hope to > agree on any kind of type assertions. Speak for yourself! :) I've tried to define this type hierarchy twice. I decided to pursue with Tim's suggestion because it provides a powerful (Turing-complete!) way of defining these types. In my opinion, typecheck.py is our type definition document for now. We'll haggle over the definitions of the classes and then we'll write a PEP that reflects our decisions. It sounds backwards but interface defintion always requires a formal syntax (IDL, ODL, PyDL). In this case our formal syntax is Python methods... -- Python: Programming the way Guido indented it. - (originated with Skip Montanaro?) From jeremy@alum.mit.edu Tue Mar 13 17:46:36 2001 From: jeremy@alum.mit.edu (Jeremy Hylton) Date: Tue, 13 Mar 2001 12:46:36 -0500 (EST) Subject: [Types-sig] Re: Revive the types sig? In-Reply-To: References: <3AACFD18.E4CE0115@ActiveState.com> <3AADC526.59DE2468@ActiveState.com> Message-ID: <15022.23804.791735.53180@w221.z064000254.bwi-md.dsl.cnc.net> It seems that what we would want is Python -> OCaml or ML2000 than Python -> ML. Are you familiar with any work of this sort? Jeremy From tim.hochberg@ieee.org Tue Mar 13 17:50:28 2001 From: tim.hochberg@ieee.org (Tim Hochberg) Date: Tue, 13 Mar 2001 10:50:28 -0700 Subject: [Types-sig] Proposed Goals PEP References: <3AACF746.9A06058F@ActiveState.com> <175b01c0ab1a$ba638e80$87740918@CX781526B> Message-ID: <1a6c01c0abe6$176a3cc0$87740918@CX781526B> ----- Original Message ----- From: "Marcin 'Qrczak' Kowalczyk" > Parameterized types are easy to support in this model, although >> the checks are potentially very expensive. [SNIP] >When a list is typechecked for the first time, items are checked >and the type is remembered. From this time the invariant is that it >contains only items of the right type. Further typechecking won't >iterate over elements but look at the stored type. Typechecking of >future items will happen at the time of their insertion. While this might be nice, I think this type of thing is an optimization the whose discussion can be deferred till later. >> Consider a parameterized list type: >I would certainly like to use real type objects and class objects in >type expressions. Unfortunately it creates a syntactic ambiguity when >user-defined parametrized types are involved: > ClassName(Type1, Type2) >This looks like a constructor call. We don't want this kind of >overloading. It looks like a constructor call because it is a constructor call. [Lots of stuff about semi-static type-checking snipped]. We've been down that road before (twice I believe), and each time the proposals have spiraled out of control and then collapsed under their own weight (to mix some metaphors). To convince people to try this path again will probably take some doing. -tim From jeremy@alum.mit.edu Tue Mar 13 17:54:03 2001 From: jeremy@alum.mit.edu (Jeremy Hylton) Date: Tue, 13 Mar 2001 12:54:03 -0500 (EST) Subject: [Types-sig] Sample declarations In-Reply-To: <200103131452.JAA32144@cj20424-a.reston1.va.home.com> References: <3AAE220A.DC92ABED@ActiveState.com> <200103131452.JAA32144@cj20424-a.reston1.va.home.com> Message-ID: <15022.24251.695911.384004@w221.z064000254.bwi-md.dsl.cnc.net> >>>>> "GvR" == Guido van Rossum writes: GvR> In short, it looks (and I am as surprised as you are!!!) like GvR> we will need to agree on the Python type hierarchy before we GvR> can ever hope to agree on any kind of type assertions. That GvR> is, unless you make the type assertions only deal with the GvR> concrete types. But that wouldn't be very useful, as there are GvR> lots of cases where you want to express that you accept any GvR> kind of sequence, mapping, file, etc. I noodled with an ML-style type inference engine for simple Python functions. One of the problems I remember running into was that most operators work for a variety of object types. If I see an expression with a "+" operator, I can't infer a lot about the operands. Even if I know one of the operands is an int, the other operand could be some other kind of number. (I was explicitly ignore an instance with an __add__ method.) It's an interesting problem, because it forces us to consider the documentation issue head on. If you write code that requirs a file-like object to have only read and readlines, what name do you give that interface? Other code might reasonably expect the file to have read, seek, and tell. Etc. We can write code quickly in Python in part because we don't have to decide in advance which methods we want to require -- and we don't have to implement every method that a builtin file has when we write a class that is a file-like object. If we want to add type declarations to argument lists, we need to document exactly what the function requires or we need to pick a useful set of standard types (supports __getitem__, mapping, mutable mapping) and fix all the code that only implements part of a particular type. Jeremy From tim.hochberg@ieee.org Tue Mar 13 17:56:37 2001 From: tim.hochberg@ieee.org (Tim Hochberg) Date: Tue, 13 Mar 2001 10:56:37 -0700 Subject: Fw: [Types-sig] Sample declarations Message-ID: <1aa201c0abe6$f3243cc0$87740918@CX781526B> I originally sent this with a gif showing a rough type hiearchy. This made the message too long and it got held by the list software. In retrospect, that image wasn't that important anyway, so I'm resending this without it. The list administrator (Paul?) can feel free to delete the original message. > [GvR] [SNIP] > > - Similar questions can be asked about streams. > > > > In short, it looks (and I am as surprised as you are!!!) like we will > > need to agree on the Python type hierarchy before we can ever hope to > > agree on any kind of type assertions. That is, unless you make the > > type assertions only deal with the concrete types. But that wouldn't > > be very useful, as there are lots of cases where you want to express > > that you accept any kind of sequence, mapping, file, etc. [Samuele Pedroni] > The point is that until now python let one be lazy and free. > Clearly we can introduce a hierarchy of so to say abstract classes, > that at least mention all the methods that a class should implement > to be a Mapping, a Sequence, ... > But my feeling is that at the moment python is more like Self or Smalltalk > an input is fine for an algorithm as long it defines just the required > "selectors" (methods) in a meaningful way. > Is that a bad practice wrt to err&doc? probably yes but allows for > flexibility and quick coding. > If for example the whole std lib will introduce typechecking that > could be a problem. Ideally, adding type checking to the standard library shouldn't make any difference; the type checking would only squawk when a callable was passed an object that was going to cause failure anyway. Whether a type hierarchy can be designed that's expressive enough to allow this without being overwhelming is another matter. I'm NOT attaching a picture I put together to try to make sense of how the sequence/mapping types fit together. Every time I look at it I end up having to make it more complicated, so I'm sure the current version is flawed, but I include it to give people something concrete to think about. Note that I've assumed that something useful comes out of the iter-sig. However, in light of Samuele's comment, I think this not the right approach. The right approach is to look to the standard library. Somebody or group of bodies should look through a sampling of the standard library and see what useful type information (in terms of method signatures) would be for sequence/mapping types. Either order (a useful set of sequence/mapping types) or chaos will emerge. For example, a quicklook at getopt.getopt (chosen randomly): def getopt(args, shortopts, longopts = []): #... args: __nonzero__ or __len__, __getitem, and __getslice__(*), members are strings shortopts: __getitem__ and __len__, members are strings longopts: string | something that can be turned into a list (__len__ and __getitem__?), members are strings. [(*) Didn't __getslice__ get subsumed by getitem?] >From this tiny little example one gets the idea that __len__ and __getitem__ (and __getslice__?) are perhaps the important elements of some type of sequencehood. However, to really get a reliable idea many more chunks of the standard library would have to be examined. We can think about type theory all we want, but it doesn't help unless it describes real code usefully. So, we might as well looking at the code. -tim From danwang@CS.Princeton.EDU Tue Mar 13 18:44:57 2001 From: danwang@CS.Princeton.EDU (Daniel Wang) Date: 13 Mar 2001 13:44:57 -0500 Subject: [Types-sig] Re: Revive the types sig? In-Reply-To: Jeremy Hylton's message of "Tue, 13 Mar 2001 12:46:36 -0500 (EST)" References: <3AACFD18.E4CE0115@ActiveState.com> <3AADC526.59DE2468@ActiveState.com> <15022.23804.791735.53180@w221.z064000254.bwi-md.dsl.cnc.net> Message-ID: Jeremy Hylton writes: > It seems that what we would want is Python -> OCaml or ML2000 than > Python -> ML. Are you familiar with any work of this sort? > > Jeremy > http://vyper.sourceforge.net/ Got, some useful code you could steal. I'm sure. The main problem however is that the semantics of Python with respect to scoping rules make this a real pain to get right. If you make some simplifying assumptions wrt scoping rules and modules this is probably a very do able hack. OCaml is probably easier to do given the polymorphic varaints and other such niceties, but I'd personally just rewrite all my code in OCaml rather than running Python code. :) (Though Python does have some nice syntax....) From danwang@CS.Princeton.EDU Tue Mar 13 19:04:38 2001 From: danwang@CS.Princeton.EDU (Daniel Wang) Date: 13 Mar 2001 14:04:38 -0500 Subject: [Types-sig] Sample declarations In-Reply-To: Jeremy Hylton's message of "Tue, 13 Mar 2001 12:54:03 -0500 (EST)" References: <3AAE220A.DC92ABED@ActiveState.com> <200103131452.JAA32144@cj20424-a.reston1.va.home.com> <15022.24251.695911.384004@w221.z064000254.bwi-md.dsl.cnc.net> Message-ID: Jeremy Hylton writes: > >>>>> "GvR" == Guido van Rossum writes: > > GvR> In short, it looks (and I am as surprised as you are!!!) like > GvR> we will need to agree on the Python type hierarchy before we > GvR> can ever hope to agree on any kind of type assertions. That > GvR> is, unless you make the type assertions only deal with the > GvR> concrete types. But that wouldn't be very useful, as there are > GvR> lots of cases where you want to express that you accept any > GvR> kind of sequence, mapping, file, etc. > > I noodled with an ML-style type inference engine for simple Python > functions. One of the problems I remember running into was that most > operators work for a variety of object types. If I see an expression > with a "+" operator, I can't infer a lot about the operands. Even if > I know one of the operands is an int, the other operand could be some > other kind of number. (I was explicitly ignore an instance with an > __add__ method.) > To get things like "+" type right, you probably want something like an "intersection type". (* read /\ as "and" *) val +: (int * int -> int) /\ (float * floa`t -> float) /\ (float * int) -> float /\ (int * float) -> float Intersection types are probably pushing the envelope for people who consider ML's type systems confusing.... but they'll solve the problems with "+" See http://www.cs.cmu.edu/afs/cs/user/fp/www/papers/icfp00.pdf and http://www.cs.cmu.edu/afs/cs/user/fp/www/talks/yale01-talk.ps which contain some pretty good examples of how intersection types work. > It's an interesting problem, because it forces us to consider the > documentation issue head on. If you write code that requirs a > file-like object to have only read and readlines, what name do you > give that interface? Other code might reasonably expect the file to > have read, seek, and tell. Etc. I'm all for "object types" i.e. types need not be associated by name. So the type of an object would just be an "anonymous interface". > We can write code quickly in Python in part because we don't have to > decide in advance which methods we want to require -- and we don't > have to implement every method that a builtin file has when we write a > class that is a file-like object. > If we want to add type declarations to argument lists, we need to > document exactly what the function requires or we need to pick a > useful set of standard types (supports __getitem__, mapping, mutable > mapping) and fix all the code that only implements part of a > particular type. Anyway, now that I think about how this whole discussion got started. i.e. lets avoid the complexity of a "real type system" and just use dynmaic checks. I think it's doomed. :) The complexity of a type system is not in how you enforce it, but deciding on how to document what you want enforced. A langauge of dynamically checked assertions has the same problem. The assertion language is just as complicated as a type system. Someone suggested that one could infer a type by examining the "check" method of an object. This actually is not as crazy as it sounds. If people are willing to program in a small subset of Python for implementing checks, these "behavioral" type checking isn't so crazy an idea. The subset would have to be a terminating side-effect free subset of python. From neelk@cswcasa.com Tue Mar 13 19:09:59 2001 From: neelk@cswcasa.com (Krishnaswami, Neel) Date: Tue, 13 Mar 2001 14:09:59 -0500 Subject: [Types-sig] Proposed Goals PEP Message-ID: Paul Prescod [mailto:paulp@ActiveState.com] wrote: > "Krishnaswami, Neel" wrote: > > > > ... > > This isn't going to help unless you have parametrized types. The > > most common case of type error in my code is when I pass in a list > > or dictionary with bogus element types. This is because usually a > > general list or dictionary of a particular shape is being used as > > a custom "type." Errors with incorrectly handling primitive types > > just doesn't seem to happen to me -- even the string/sequence > > thing basically never bites me. > > How would you propose to implement type checking of potentially > heterogenous lists against a potentially complex type (i.e. one with > many subtypes) in an efficient manner? There are limits to the > performance price I am willing to ask users to incur for better type > checking! Off the top of my head: First, we could under the covers add fields in the list and dictionary objects noting what key and element types they have. When a value is created with a type declaration, then the key and element type fields are filled in, and whenever __setitem__ is called, a typecheck can be done to make sure that the modification is ok. In addition to signalling an error at the point the invariant breaks, this also ensures that the precise type is never wrong. Eg (with bogus example syntax): >>> d :: {Int: String} = {3: "foo", 4: "bar"} >>> d[17] = 42 TypeError: d is of type {Int: String}, assigned an integer. If a dictionary d is created without a type declaration, then the key and element fields are not set. (More likely, we'd want a new type like Object or Top or Any to represent arbitrary values.) When a function with type declarations receives a value, it can first check to see if the value has a precise type. If it does, then the typecheck is fast. It's only if the value's type is less precise than the declaration that you need to walk all the elements of the structure. Dynamically-checked code doesn't need to do this at all, leading to the following speed hierarchy: 1: untyped code fed values with any types 2: typed code fed values with precise types 3: typed code fed values with imprecise types Like I said, this is off the top of my head. There are obvious refinements, that might or might not be useful: keeping track of the precise type even of untyped lists and dictionaries. I think it's probably okay to special-case lists and dictionaries, since they are the most common source of such errors. Here's an implemenation to give you the idea: import UserDict class TypedDict(UserDict.UserDict): def __init__(self, key_type, val_type, data=None): self.key_type = key_type self.val_type = val_type self.data = {} if data != None: for (k, v) in data.items(): self[k] = v def __setitem__(self, k, v): if isinstance(k, self.key_type) and isinstance(v, self.val_type): self.data[k] = v else: raise TypeError, ("key %s, val %s" % (str(k), str(v))) class TypedDict(UserDict.UserDict): def __init__(self, key_type, val_type, data=None): self.key_type = key_type self.val_type = val_type self.data = {} if data != None: for (k, v) in data.items(): self[k] = v def __setitem__(self, k, v): if isinstance(k, self.key_type) and isinstance(v, self.val_type): self.data[k] = v else: raise TypeError, ("key %s, val %s wrong type" % (str(k), str(v))) def check(self, key_type, val_type): if self.key_type == key_type and self.val_type == val_type: return 1 else: ans = 1 for k, v in self.data.items(): if not (isinstance(k, key_type) and isinstance(v, val_type)): ans = 0 break return ans > Also note that even without parameterized type checks, you would often > catch your error closer to where it occurred than you would > today. Yes, you might not catch it at the point you pass in the list > but you would catch it as soon as you passed one of the arguments to > a sub-function. I'm unconvinced -- collections are where typechecking is most needed, and where it is least provided. (Excepting ML and Haskell.) -- Neel Krishnaswami neelk@cswcasa.com From paulp@ActiveState.com Tue Mar 13 19:12:32 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Tue, 13 Mar 2001 11:12:32 -0800 Subject: [Types-sig] Sample declarations References: <3AAE220A.DC92ABED@ActiveState.com> <200103131452.JAA32144@cj20424-a.reston1.va.home.com> <15022.24251.695911.384004@w221.z064000254.bwi-md.dsl.cnc.net> Message-ID: <3AAE7120.E798D15F@ActiveState.com> Jeremy Hylton wrote: > >... > > If we want to add type declarations to argument lists, we need to > document exactly what the function requires or we need to pick a > useful set of standard types (supports __getitem__, mapping, mutable > mapping) and fix all the code that only implements part of a > particular type. I think we will fail if we try to be too prescriptive and tight in our definitions. I assert that the primary goal of the system is to check for accidental passing of one type for another then the fact that you might pass an incompletely implemented class is merely a logic error like dividing by 0. That's the beauty of type systems: anything you don't want to implement (for whatever reason) ends up being called a logic/runtime error. And to a certain extent this is ALWAYS going to be the situation. What happens in Java when I don't feel like implementing a method that I don't think I need? I just put a NotImplementedAssertion in there and check that it never gets hit at runtime. How is that better than simply choosing not to implement the method so that the Python *runtime* raises the NotImplementedAssertion? I propose that we try to work out a set of interfaces for common protocols. If it starts to get out of hand in subtypes of subtypes of union types of subtypes then we merely back off to a situation more similar to the status quo. I think that there are two things we need to make it work a little bit better: 1. A way for instances to say "HEY! I'm a mapping, just like that C extension object over there" It is a little weird that there is a tp_mapping slot but no __i_am_a_mapping__ flag. __interfaces__ is the obvious mechanism but it's been held up in committee for several administrations.... 2. Documentation on the *general* requirements for mapping types. Or perhaps the *minimal* requirements for mapping types. They can choose to go beyond and authors can choose to use more at their own risk. If we can also find mutually agreeable formal definitions for the protocols then that's great. But if we back-off to the position that anything that claims to be a mapping passes a type-check for Mapping, that's acceptable too. This is actually how the C API works today. How do we know if something is a sequence? All it needs to do is claim that it is. Not every sequence operation is guaranteed to work for every sequence-claiming object. It's incredibly loose and would give anyone coming to the language from a strongly-typed language the heebie-jeebies. But it seems to work! >>> map(None, []) [] >>> map(None,{}) Traceback (most recent call last): File "", line 1, in ? TypeError: list() argument must be a sequence Could I break the map function by passing in a "weird" sequence? Yes. Life goes on. Summary: agreed-upon, formal definitions of protocols are NOT a requirement for success. -- Python: Programming the way Guido indented it. - (originated with Skip Montanaro?) From paulp@ActiveState.com Tue Mar 13 19:15:01 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Tue, 13 Mar 2001 11:15:01 -0800 Subject: [Types-sig] Sample declarations References: <3AAE220A.DC92ABED@ActiveState.com> <200103131452.JAA32144@cj20424-a.reston1.va.home.com> <15022.24251.695911.384004@w221.z064000254.bwi-md.dsl.cnc.net> Message-ID: <3AAE71B5.93C49CC4@ActiveState.com> Daniel Wang wrote: > >... > > Anyway, now that I think about how this whole discussion got started. > i.e. lets avoid the complexity of a "real type system" and just use dynmaic > checks. I think it's doomed. :) The complexity of a type system is not in > how you enforce it, but deciding on how to document what you want enforced. > A langauge of dynamically checked assertions has the same problem. The > assertion language is just as complicated as a type system. The assertion language is incredibly complex compared to IDL. It is Python. But the beauty of using Python as an assertion language is that Python programmers already know Python! > Someone suggested that one could infer a type by examining the "check" > method of an object. This actually is not as crazy as it sounds. If people > are willing to program in a small subset of Python for implementing checks, > these "behavioral" type checking isn't so crazy an idea. The subset would > have to be a terminating side-effect free subset of python. I like the idea. I'll leave it to a PhD student to define that subset. -- Python: Programming the way Guido indented it. - (originated with Skip Montanaro?) From paulp@ActiveState.com Tue Mar 13 19:42:51 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Tue, 13 Mar 2001 11:42:51 -0800 Subject: [Types-sig] Proposed Goals PEP References: Message-ID: <3AAE783B.2C6C5A76@ActiveState.com> "Krishnaswami, Neel" wrote: > >... > type is never wrong. > > Eg (with bogus example syntax): > > >>> d :: {Int: String} = {3: "foo", 4: "bar"} > >>> d[17] = 42 > TypeError: d is of type {Int: String}, assigned an integer. I'd rather not bring in declarations of object types. People are going to confuse that with declarations of VARIABLE types (which they will confuse with declarations of PARAMATER types). Plus, we are going to get flamed for changing the language for a performance optimization. Finally, the whole system gets weird when you start subtyping. We would need to do a potentially expensive issubclass on each insert and then, when you pass the object to a function, if the function's types do not exactly line up with yours, your performance goes all to hell because it may need to do the whole thing again. In general, I still think parametric type checking is something I want to defer. I like the fact that the executable types proposal grows naturally into those kinds of type checking without requiring US to specify them as part of standard Python. If you want to write code that is paranoid and slow, you can do it but average Python programmers would not accidently change the whole runtime behavior of their program by adding a few declarations. And how does a list of tuples of a list of tuples work? I have to generate strongly typed objects at every level? I think the costs are outweighing the benefits...maybe StringSequence is such a common case that we should have a special type for it. -- Python: Programming the way Guido indented it. - (originated with Skip Montanaro?) From danwang@CS.Princeton.EDU Tue Mar 13 19:40:03 2001 From: danwang@CS.Princeton.EDU (Daniel Wang) Date: 13 Mar 2001 14:40:03 -0500 Subject: [Types-sig] Sample declarations In-Reply-To: Paul Prescod's message of "Tue, 13 Mar 2001 11:15:01 -0800" References: <3AAE220A.DC92ABED@ActiveState.com> <200103131452.JAA32144@cj20424-a.reston1.va.home.com> <15022.24251.695911.384004@w221.z064000254.bwi-md.dsl.cnc.net> <3AAE71B5.93C49CC4@ActiveState.com> Message-ID: Paul Prescod writes: {stuff deleted} > > Someone suggested that one could infer a type by examining the "check" > > method of an object. This actually is not as crazy as it sounds. If people > > are willing to program in a small subset of Python for implementing checks, > > these "behavioral" type checking isn't so crazy an idea. The subset would > > have to be a terminating side-effect free subset of python. > > I like the idea. I'll leave it to a PhD student to define that subset. Acutally, if all you really care about is "earlier errors" for debugging purposes, and if you can convince a PhD student to do some work for you. There is a "simple" and completely automatic way to get earlier errors. given a method or function body def foo (....): random code... automatically compute a "type assertion" function that basically simulates the behavior of the random code with respect to type errors i.e. if (b) then primop1 (* doesn't care what type x is *) primop2(x) (* may fail if x is not an int *) else primop2(x) (* may fail if x is not an float *) foo(x) (* call the function foo *) get's converted to.. if(debug) then if b then (checkInt(x)) else checkFloat(x) checkFoo(x) (* call foo's check function *) (* original code *) or to put it another way, do some simple program analysis and "hoist" type errors to the earliest possible point in the computation. This can be completely automatic, and in the worst case you leave your program alone if it isn't smart enough to figure out how to hoist errors earlier then they already occur. Also, if the assertions are really accurate, an optimizing python compiler can use that fact to generate good code. Also, this doesn't require any language extensions, just some fancy compiler hacking and program analysis. Even something very dumb might do the trick. The first hack, would be to hoist type assertions to the beginning of every basic block. Then the next step would be to merge and hoist type assertions based on control flow information. (This idea should work for Java and Scheme too... I'd be surprised if someone hasn't tried this before.... but you never know... most people don't think program analysis is easy....) From michel@digicool.com Tue Mar 13 21:02:22 2001 From: michel@digicool.com (Michel Pelletier) Date: Tue, 13 Mar 2001 13:02:22 -0800 (PST) Subject: [Types-sig] Interface PEP Message-ID: Hi folks, Before I release this more broadly, I'd like a few of you to take a look at it for any obvious error. http://www.zope.org/Members/michel/InterfacesPEP/PEP.txt I will be posting it to the python lists and soliciting a PEP number later this week. I just wanted to drop this here to see what y'all thought. I intentionally left out any part of the proposal that satified my previous "Interfaces can do type checking" assertion. I still belive they can, but it would require a lot more thought about Python types and checking in general, and such an extension could be added at a later date to this PEP. As an aside, this might be the first PEP that proposes an extention to python that describes that extension with the extension it proposes. ;) Thanks, -Michel From qrczak@knm.org.pl Tue Mar 13 20:25:56 2001 From: qrczak@knm.org.pl (Marcin 'Qrczak' Kowalczyk) Date: 13 Mar 2001 20:25:56 GMT Subject: [Types-sig] Proposed Goals PEP References: <3AAE783B.2C6C5A76@ActiveState.com> Message-ID: Tue, 13 Mar 2001 11:42:51 -0800, Paul Prescod pis= ze: > And how does a list of tuples of a list of tuples work? I have to > generate strongly typed objects at every level? Neel Krishnaswami proposed essentially the same concept as me, so I will defend the concept. Among these types only lists need to store the type of items (a single object for the whole list). A dict stores two types. A tuple has a fixed shape (length and types of elements). Since it's immutable, it does not store the shape explicitly. The shape is conceptually derived from the tuple's contents. We don't need to extend the representation of tuple to let it be typed, only define how to typecheck a tuple. A type constraint which defines a tuple shape, applied to a tuple object, just pushes declared types of items to actual items, after checking that the length matches. If we feel the need, we may define a more flexible ways to describe tuple shapes, e.g. allowing unspecified number of items of a specified type. But I'm not doing that. This may only be considered after the type system is ready and working, and we are happy with its basics but want this flexibility. > maybe StringSequence is such a common case that we should have a > special type for it. No, there will be no such special cases at all. --=20 __("< Marcin Kowalczyk * qrczak@knm.org.pl http://qrczak.ids.net.pl/ \__/ ^^ SYGNATURA ZAST=CAPCZA QRCZAK From qrczak@knm.org.pl Tue Mar 13 20:52:56 2001 From: qrczak@knm.org.pl (Marcin 'Qrczak' Kowalczyk) Date: 13 Mar 2001 20:52:56 GMT Subject: [Types-sig] Sample declarations References: <3AAE220A.DC92ABED@ActiveState.com> <200103131452.JAA32144@cj20424-a.reston1.va.home.com> <15022.24251.695911.384004@w221.z064000254.bwi-md.dsl.cnc.net> <3AAE71B5.93C49CC4@ActiveState.com> Message-ID: 13 Mar 2001 14:40:03 -0500, Daniel Wang pisze: > given a method or function body > def foo (....): > random code... >=20 > automatically compute a "type assertion" function that basically > simulates the behavior of the random code with respect to type errors This will never work well in Python. Python is too dynamic and the behavior of libraries wrt. types is too irregular to be inferred statically from usage (perhaps except most simple cases, which is impractical). Sorry, a simple statement of spam.spam(spam, spam) generates a complex assertion that the object to which spam is bound, whatever it is at this point, has the attribute 'spam', or at least is able to produce it, which is a callable object, whatever type it is, which can accept two arguments, not necessarily only two, whose types are required to be some unknown supertypes of the type of the object to which spam is bound, assuming that their requirements can be described in terms of supertypes at all. You can't even combine assertions about the 'spam' attribute derived from two calls, because nothing prevents rebinding the attribute at any point. Type inference works very well in SML, OCaml, Haskell, Clean and Miranda. These languages have a simple model of behavior of basic objects and simple type rules. Libraries are designed according to these rules. Polymorphism is present only in resticted ways, quite different from the Python way. The result is that these languages allow to program with almost no explicit type annotations, yet fully protected from type errors and not too limited, if you agree to think according to their way. But it's far from Python, and not compatible with existing Python's style at all. At most there can be (I hope) a soft type system, with optional assertion-like type annotations. It would even allow applying optimizations by a smart interpreter: fortunately rebinding of local variables is restricted to the body of the given function, so the interpreter can sometimes be sure that a local variable is indeed always an integer. --=20 __("< Marcin Kowalczyk * qrczak@knm.org.pl http://qrczak.ids.net.pl/ \__/ ^^ SYGNATURA ZAST=CAPCZA QRCZAK From danwang@CS.Princeton.EDU Tue Mar 13 21:42:58 2001 From: danwang@CS.Princeton.EDU (Daniel Wang) Date: 13 Mar 2001 16:42:58 -0500 Subject: [Types-sig] Sample declarations In-Reply-To: qrczak@knm.org.pl's message of "13 Mar 2001 20:52:56 GMT" References: <3AAE220A.DC92ABED@ActiveState.com> <200103131452.JAA32144@cj20424-a.reston1.va.home.com> <15022.24251.695911.384004@w221.z064000254.bwi-md.dsl.cnc.net> <3AAE71B5.93C49CC4@ActiveState.com> Message-ID: qrczak@knm.org.pl (Marcin 'Qrczak' Kowalczyk) writes: > 13 Mar 2001 14:40:03 -0500, Daniel Wang pisze: > > > given a method or function body > > def foo (....): > > random code... > >=20 > > automatically compute a "type assertion" function that basically > > simulates the behavior of the random code with respect to type errors > > This will never work well in Python. Python is too dynamic and the > behavior of libraries wrt. types is too irregular to be inferred > statically from usage (perhaps except most simple cases, which is > impractical). Ahh.. but you missed the point... in the worst case, the system does absolutely nothing. i.e. if it can't figure out how to move type errors earlier, it just does nothing. > > Sorry, a simple statement of > spam.spam(spam, spam) > generates a complex assertion that > the object to which spam is bound, whatever it is at this point, > has the attribute 'spam', or at least is able to produce it, > which is a callable object, whatever type it is, which can accept > two arguments, not necessarily only two, whose types are required > to be some unknown supertypes of the type of the object to which > spam is bound, assuming that their requirements can be described > in terms of supertypes at all. > > You can't even combine assertions about the 'spam' attribute derived > from two calls, because nothing prevents rebinding the attribute at > any point. > Type inference works very well in SML, OCaml, Haskell, Clean and > Miranda. These languages have a simple model of behavior of basic > objects and simple type rules. Libraries are designed according to > these rules. Polymorphism is present only in resticted ways, quite > different from the Python way. I think the real problem is the possibility that identifiers can get rebound at almost any point in time. Other than that, python's type rules aren't that much more complex. Also, I'm not attempting to suggest we do type inference, just "type check hoisting". > The result is that these languages allow to program with almost no > explicit type annotations, yet fully protected from type errors and > not too limited, if you agree to think according to their way. But > it's far from Python, and not compatible with existing Python's style > at all. Again, I think you misunderstood the idea... in the limit it leaves your Python program alone. If it can move the type error earlier it does. I don't promise much in the way of ruling out type errors. I'm just suggesting a way to make sure the type errors are reported earlier, if they occur. > At most there can be (I hope) a soft type system, with optional > assertion-like type annotations. It would even allow applying > optimizations by a smart interpreter: fortunately rebinding of local > variables is restricted to the body of the given function, so the > interpreter can sometimes be sure that a local variable is indeed > always an integer. Soft typing has it's limitations with respect to separate compilation. What i'm suggesting is more modular, will work with any Python program. I'm not so sure it will however be useful in general or not... someone needs to bite the bullet and do an experiment by hand..... If someone could point me to a reference for the Python bytecode language that would be a plus. Doing this at the level of bytecodes will be much easier than at the source code level. From paulp@ActiveState.com Tue Mar 13 22:31:55 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Tue, 13 Mar 2001 14:31:55 -0800 Subject: [Types-sig] Sample declarations References: <3AAE220A.DC92ABED@ActiveState.com> <200103131452.JAA32144@cj20424-a.reston1.va.home.com> <15022.24251.695911.384004@w221.z064000254.bwi-md.dsl.cnc.net> <3AAE71B5.93C49CC4@ActiveState.com> Message-ID: <3AAE9FDB.9BD4C737@ActiveState.com> Daniel Wang wrote: > >... > > If someone could point me to a reference for the Python bytecode language > that would be a plus. Doing this at the level of bytecodes will be much > easier than at the source code level. Here you go: http://velocity.activestate.com/docs/ActivePython/lib/bytecodes.html -- Python: Programming the way Guido indented it. - (originated with Skip Montanaro?) From barry@digicool.com Tue Mar 13 22:45:11 2001 From: barry@digicool.com (Barry A. Warsaw) Date: Tue, 13 Mar 2001 17:45:11 -0500 Subject: [Types-sig] A nice alternative view about static types Message-ID: <15022.41719.470736.909236@anthem.wooz.org> Cameron Laird forwarded this article to python-list as a nice plug for Python, but I think it has some useful insights w.r.t. the current discussions on this list. http://forums.itworld.com/webx?14@139.drRfa36vgId^0@.ee6eed4/11 -Barry From barry@digicool.com Tue Mar 13 23:08:22 2001 From: barry@digicool.com (Barry A. Warsaw) Date: Tue, 13 Mar 2001 18:08:22 -0500 Subject: [Types-sig] Interface PEP References: Message-ID: <15022.43110.167035.671516@anthem.wooz.org> >>>>> "MP" == Michel Pelletier writes: MP> Before I release this more broadly, I'd like a few of you to MP> take a look at it for any obvious error. MP> http://www.zope.org/Members/michel/InterfacesPEP/PEP.txt BTW, I had to grab it with wget because I think the server isn't setting the document type correctly. It was just run-on text in my browser. -Barry From michel@digicool.com Wed Mar 14 00:20:40 2001 From: michel@digicool.com (Michel Pelletier) Date: Tue, 13 Mar 2001 16:20:40 -0800 (PST) Subject: [Types-sig] Interface PEP In-Reply-To: <15022.43110.167035.671516@anthem.wooz.org> Message-ID: On Tue, 13 Mar 2001, Barry A. Warsaw wrote: > > >>>>> "MP" == Michel Pelletier writes: > > MP> Before I release this more broadly, I'd like a few of you to > MP> take a look at it for any obvious error. > > MP> http://www.zope.org/Members/michel/InterfacesPEP/PEP.txt > > BTW, I had to grab it with wget because I think the server isn't > setting the document type correctly. It was just run-on text in my > browser. Really? It's content type is text/plain. It looks fine in Netscape, Mozilla, Links and Lynx. What browser are you using (I can only guess...). Anyone else have a problem with it? -Michel From barry@digicool.com Tue Mar 13 23:28:45 2001 From: barry@digicool.com (Barry A. Warsaw) Date: Tue, 13 Mar 2001 18:28:45 -0500 Subject: [Types-sig] Interface PEP References: <15022.43110.167035.671516@anthem.wooz.org> Message-ID: <15022.44333.335069.355747@anthem.wooz.org> >>>>> "MP" == Michel Pelletier writes: MP> Really? It's content type is text/plain. It looks fine in MP> Netscape, Mozilla, Links and Lynx. What browser are you using MP> (I can only guess...). Naw, I'm pretty conservative with my web browser. Just NS 4.74 on Linux. Weird. -B From sverker.is@home.se Tue Mar 13 23:43:17 2001 From: sverker.is@home.se (Sverker Nilsson) Date: Wed, 14 Mar 2001 00:43:17 +0100 Subject: [Types-sig] Interface PEP References: Message-ID: <3AAEB095.2119C723@home.se> Michel Pelletier wrote: > > Anyone else have a problem with it? I had problems too, using Netscape 3.01 on Linux. Could read it with "view document source" only. Now that I tried again it worked anyway! Maybe something changed?! Another detail I found: At one place you use this syntax to say you extend interfaces: interface FishMarketInterface(CountFishInterface, ColorFishInterface) But on another place: interface AttributeInterface extends InterfaceBaseInterface I was wondering if that meant the same or something different. Sverker From michel@digicool.com Wed Mar 14 00:42:46 2001 From: michel@digicool.com (Michel Pelletier) Date: Tue, 13 Mar 2001 16:42:46 -0800 (PST) Subject: [Types-sig] Interface PEP In-Reply-To: <3AAEB095.2119C723@home.se> Message-ID: On Wed, 14 Mar 2001, Sverker Nilsson wrote: > Michel Pelletier wrote: > > > > Anyone else have a problem with it? > > I had problems too, using Netscape 3.01 on Linux. Could read > it with "view document source" only. > > Now that I tried again it worked anyway! Maybe something changed?! Well, the moon moved a bit in its orbit. That was probably it. > Another detail I found: At one place you use this syntax to say > you extend interfaces: > > > interface FishMarketInterface(CountFishInterface, ColorFishInterface) > > > But on another place: > > > interface AttributeInterface extends InterfaceBaseInterface > > > I was wondering if that meant the same or something different. Oops, I changed it to the former. Pay no attention to the man behind the curtain. -Michel From paulp@ActiveState.com Tue Mar 13 23:48:59 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Tue, 13 Mar 2001 15:48:59 -0800 Subject: [Types-sig] Interface PEP References: Message-ID: <3AAEB1EB.A03DED06@ActiveState.com> Michel Pelletier wrote: > > ... > > Really? It's content type is text/plain. It looks fine in Netscape, > Mozilla, Links and Lynx. What browser are you using (I can only > guess...). > > Anyone else have a problem with it? I swear I had a problem at one point with NS 4.76 but now it is working fine. Did you change something? I remember switching from NS to IE because NS was showing it funny. But now it isn't. -- Python: Programming the way Guido indented it. - (originated with Skip Montanaro?) From michel@digicool.com Wed Mar 14 00:49:53 2001 From: michel@digicool.com (Michel Pelletier) Date: Tue, 13 Mar 2001 16:49:53 -0800 (PST) Subject: [Types-sig] Interface PEP In-Reply-To: <3AAEB1EB.A03DED06@ActiveState.com> Message-ID: On Tue, 13 Mar 2001, Paul Prescod wrote: > Michel Pelletier wrote: > > > > ... > > > > Really? It's content type is text/plain. It looks fine in Netscape, > > Mozilla, Links and Lynx. What browser are you using (I can only > > guess...). > > > > Anyone else have a problem with it? > > I swear I had a problem at one point with NS 4.76 but now it is working > fine. Did you change something? I remember switching from NS to IE > because NS was showing it funny. But now it isn't. Hmm. Oh well, at least it works now. I'm pretty certain I didn't change anything with it, other than edit its contents through FTP. -Michel From paulp@ActiveState.com Tue Mar 13 23:55:01 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Tue, 13 Mar 2001 15:55:01 -0800 Subject: [Types-sig] Interface PEP References: Message-ID: <3AAEB355.285C733@ActiveState.com> Michel Pelletier wrote: > > ... > > Hmm. Oh well, at least it works now. I'm pretty certain I didn't change > anything with it, other than edit its contents through FTP. I bet you changed the line-ends by accident. -- Python: Programming the way Guido indented it. - (originated with Skip Montanaro?) From pedroni@inf.ethz.ch Wed Mar 14 00:04:54 2001 From: pedroni@inf.ethz.ch (Samuele Pedroni) Date: Wed, 14 Mar 2001 01:04:54 +0100 Subject: [Types-sig] Sample declarations References: <3AAE220A.DC92ABED@ActiveState.com> <200103131452.JAA32144@cj20424-a.reston1.va.home.com> <15022.24251.695911.384004@w221.z064000254.bwi-md.dsl.cnc.net> <3AAE71B5.93C49CC4@ActiveState.com> Message-ID: <01c901c0ac1a$67424f60$f979fea9@newmexico> Hi. [Daniel Wang] > > I think the real problem is the possibility that identifiers can get > rebound at almost any point in time. Other than that, python's type rules > aren't that much more complex. Also, I'm not attempting to suggest we do > type inference, just "type check hoisting". > > Again, I think you misunderstood the idea... in the limit it leaves your > Python program alone. If it can move the type error earlier it does. I don't > promise much in the way of ruling out type errors. I'm just suggesting a way > to make sure the type errors are reported earlier, if they occur. > If I understood well your suggesting that some code analysis given the following: def f(a,b): if b>=0: return a[-1] else: return a[1] could infere we could for example check param a for "having" __getitem__ directly at the beginning of f. Yes it could. I don't think that we can in general infere checking for being a float or an int. You're saying that a problem is so to say the excess of dynamism of python, this is true (also wrt opt). But I continue not to understand why languages like ML are cited as an example, they have polymorphism in the form of parametric types. Python has just crude polymorphism (like Smalltalk or Self): def add(a,b): return a+b any parameters that support + (int,float, ... exposing __add__ or __radd__) can do the job here and are ok, if we add type checks or annotations they fix programmer intentions (about usage) or supply information about the context, from the correctness viewpoint what is really important are the (calling&called) contexts, we can infere nothing about them because python is too dynamic, has reflective features and dynamic loading, we cannot make any closed world assumption and in any case the whole call graph problem is difficult and would require type inference (as everybody knows in the case of python and similar the two make together a fix-point problem). This context problem limit a lot the amount of error checking that one can hoist... regards, Samuele Pedroni. From tim.one@home.com Wed Mar 14 00:50:25 2001 From: tim.one@home.com (Tim Peters) Date: Tue, 13 Mar 2001 19:50:25 -0500 Subject: [Types-sig] Proposed Goals PEP In-Reply-To: <3AAE0E4A.D7E7A58E@ActiveState.com> Message-ID: [Paul Prescod] > ... > Those are the wrong people to worry about. They say any type > declarations as cruft anyhow. But there are some people who want to use > this information in static ways. I had hoped not to get in their way! Paul, you're free to let this paralyze your efforts again, but aren't you tired of it yet <0.3 wink>? Nobody in practice steps on another module's globals by *accident*, so worrying about people stomping on the names in a std typecheck module is worrying about actively hostile users. Who cares? If you must, fine, just *say* that in the presence of the "please check types" flag, the behavior is undefined if any but the std typecheck module is first on the path, or if anyone farts around with its contents. Then in *reality* typecheck.Integer etc would be as safe as reserved names, and anyone going out of their way to break it deserves whatever they labored to break. imaginary-problems-don't-need-tech-solutions-ly y'rs - tim From guido@digicool.com Wed Mar 14 01:21:23 2001 From: guido@digicool.com (Guido van Rossum) Date: Tue, 13 Mar 2001 20:21:23 -0500 Subject: [Types-sig] Sample declarations In-Reply-To: Your message of "Tue, 13 Mar 2001 09:18:12 PST." <3AAE5654.60539291@ActiveState.com> References: <3AAE220A.DC92ABED@ActiveState.com> <200103131452.JAA32144@cj20424-a.reston1.va.home.com> <3AAE5654.60539291@ActiveState.com> Message-ID: <200103140121.UAA02571@cj20424-a.reston1.va.home.com> > > - Your test for number assumes that something can be converted to > > complex. That may not be true. > > What's an appropriate definition for number? operator.isNumberType()? No, the situation is approximately as murly as for sequences, mappings or files. > > - Your test for Integer includes float/complex numbers with integral > > values; that doesn't work, e.g. a[1.0] raises TypeError if a is a > > sequence. > > And yet this works: > > open("c:\\autoexec.bat", "r", 5.0).read() > > Is it by design that these two things work differently? Or is it an > implementation quirk? It's an implementation quirk that this works -- it really shouldn't! Note that even 5.1 as the buffer size works -- and *that* *really* shouldn't be allowed. The deep reason is that if a type supports conversion to integer, you don't know if it's widening from a smaller type of int, or narrowing from e.g. float. > And if it is an implementation quirk then maybe we should use this as an > opportunity to make the language more consistent by loosening up some > type checks. Or tightening, depending on what feels right. If and when the numeric unification is introduced, I'm all for allowing 5.0 as buffer size (but disallowing 5.1). Until that all is in place, I'm more comfortable with requiring that a Python int is given when the C code needs a C int. > My impression is that PyArg_ParseTuples is pretty flexible and it made > sense for the Python equivalent of PyArg_ParseTuples to be as similar as > possible. Of course one big definition between PyArg_ParseTuples and > __implementedby__ is that the former actually coerces its args and the > latter only checks them. Maybe this is a design flaw.... Yes, the flaw is in the current PyArg_ParseTuple() implementation. > If there is an obvious way to coerce an object to a type, we should > probably use it. This will make the whole system's behavior more > predictable. But this doesn't work: > > coerce(5, types.FloatType) > > In general I don't know how to try to coerce objects to types. I only > know how to coerce two objects to some common type. The coerce() function is irrelevant here -- it takes two objects and returns a tuple of the two coerced to a common type. What you want is a cast. We have cast functions named after the concrete types, e.g. int(), float(), long(), etc. > > - Your test for Float values that accepts ints is also bad news, at > > least until integer division is fixed. You may want to define a > > separate type IntOrFloat, for those cases where the programmer > > asserts that either type is fine, and another IntOrFloatOrComplex. > > Not entirely happy with that design!!! I'd rather change the language > (which we already intend to fix) rather than hack up the type system. OK, but then the burden is on you to propose a way to fix the language that can be introduced without breaking existing code -- or else you will have to walk the slow and tedious way of all incompatible language changes, and that would get in the way of experimenting with type annotations "soon". > > > def findInterface(iface, obj): > > > ifaces = getattr(obj, "__interfaces__", None) > > > # don't make this mutually recursive on > > > # Sequence.__implementedby__! > > > if type(ifaces) in (types.ListType, types.TupleType): > > > return iface in ifaces # multiple interfaces case > > > else: > > > return iface is ifaces # single interface case > > > > This helper function is steeped in irrelevant implementation detail! > > I'm implementing something! I have to put in all of the details required > to make it work. It's supposed to be a prototype not pseudocode. Hm, everything else appeared so simplistic that it looked more as if it was pseudo-code. But I see your point. Keep trying! > > - Your assumptions about sequences, mappings etc. are also pretty > > naive. operator.isSequenceType() and .isMappingType() return true > > for all class instances regardless of what they actually implement, > > and should this be avoided. > > I had an explicit check for InstanceType-ness but accidentally deleted > it. I thought I had incorporated it into findInterface. I'll put it back > in. > > Here's Sequence, fixed up: > > class _sequence: > "Any kind of sequence" > def __implementedby__(self, obj): > if findInterface(self, obj): # if an object claims to support > return 1 # the protocol, we trust it! > elif type(obj)==types.InstanceType: # no safe way to know if an > instance > return 0 # supports the protocol > else: > return operator.isSequenceType(obj) # safe for C-types > Sequence=_sequence() Or for instance you could assume it's a sequence if it has __getitem__. This guesses wrong when it wants to be a mapping, but I believe it's better to accept too much than to reject too much here (since it's clear that we aren't going to get it right 100% anyway -- we don't want to break correct code). > > - One way to go would be to have explicit ways to say "x is any > > Sequence", "x is a List", "x is any Mapping", "x is a Dictionary". > > We do: > > def foo(x: Sequence, x:ListType, x:Mapping, x:DictionaryType): ... OK. > I haven't implemented the part that checks type objects (or class > objects, for that matter). Here's a new version with an implementation > for that: > > def experimentalAssertTypes(dict): > # this is VERY ROUGH > # for one thing it should use variable names from the caller, > # not a passed in dictionary! Can I make a comment on these comments? You seem to be using comments only when there's something not 100% kosher in your code. An implied "XXX". :-) I'd appreciate to see more comments that actually explain what you are doing and why, rather than what you are *not* (yet) doing! (Most of JPython was once written using this convention -- while it was clean code, it was very hard to read for the second developer...) > for val, typ in dict.items(): > if type(typ) in (types.ClassType, types.TypeType): > if not isinstance(val, typ): > raise TypeError(val, typ) > elif typ.__implementedby__(val): > continue > else: > raise TypeError(val, typ) > > > - But note again that Sequence-ness and Mapping-ness are ill-defined. > > I agree. This comes back to the issue of interfaces. I think that the DC > guys have made some progress on a standard type hierarchy, haven't they? Not that I know of. I've seen an UML diagram purporting to be "the Python type herarchy" but it has all of the same flaws your first attempt exposed, and a lot more time was spent. :-( > > In short, it looks (and I am as surprised as you are!!!) like we will > > need to agree on the Python type hierarchy before we can ever hope to > > agree on any kind of type assertions. > > Speak for yourself! :) I've tried to define this type hierarchy twice. I > decided to pursue with Tim's suggestion because it provides a powerful > (Turing-complete!) way of defining these types. In my opinion, > typecheck.py is our type definition document for now. We'll haggle over > the definitions of the classes and then we'll write a PEP that reflects > our decisions. OK, fair enough! > It sounds backwards but interface defintion always requires a formal > syntax (IDL, ODL, PyDL). In this case our formal syntax is Python > methods... Good luck! --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@home.com Wed Mar 14 01:31:41 2001 From: tim.one@home.com (Tim Peters) Date: Tue, 13 Mar 2001 20:31:41 -0500 Subject: [Types-sig] Sample declarations In-Reply-To: <200103131452.JAA32144@cj20424-a.reston1.va.home.com> Message-ID: [Guido, rediscovers that "sequence" and "mapping" etc are ill-defined] > ... > Before you can meaningfully discuss the Sequence check code, you > should make up your mind when you start calling something a > sequence. Clearly it needs __getitem__ (let's limit ourselves to > class instances). Not clear to me: the instant the new iterator protocol gets added to the language, over half my "sequences" throw away their artificial __getitem__ methods, despite that they would work fine in oodles of code that expects "a sequence". def sum(seq): sum = 0L for s in seq: sum += s return sum That should work dandy to add up the ints attached to the nodes of my (say) binary tree "seq", and a binary tree has no natural need to implement __getitem__. > ... > In short, it looks (and I am as surprised as you are!!!) like we will > need to agree on the Python type hierarchy before we can ever hope to > agree on any kind of type assertions. Were you on vacation the last two times this SIG was alive ? We've never managed to agree on useful definitions for Python's "folklore protocols" -- and that wasn't for lack of trying. Old news: adding type checks subtracts from flexibility, unless the type checks are at the level of individual operations (Getitem_able, TwoIndexSlice_able, Has_key_able, Iter_able, etc), in which case type checks can add enormous tedium, and/or enormous confusion as N programmers each group the lowest-level capabilities in their own unique ways. But Paul shouldn't let even that stop him. While the folklore protocols are ineffable this way, a great many useful typechecks in practice will be of very simple isinstance(object, types.IntType) and isinstance(object, SomeClassName) forms. For all the rest, we make up something more or less arbitrary. A saving grace is that many applications of the folklore protocols truly require only a single method, so that e.g. WriteString_able is a perfectly adequate typecheck for the many routines that require only .write(String) of a "file-like object". panic-is-premature-albeit-prudent-ly y'rs - tim From michel@digicool.com Wed Mar 14 02:46:16 2001 From: michel@digicool.com (Michel Pelletier) Date: Tue, 13 Mar 2001 18:46:16 -0800 (PST) Subject: [Types-sig] Sample declarations In-Reply-To: <200103140121.UAA02571@cj20424-a.reston1.va.home.com> Message-ID: On Tue, 13 Mar 2001, Guido van Rossum wrote: > > I agree. This comes back to the issue of interfaces. I think that the DC > > guys have made some progress on a standard type hierarchy, haven't they? > > Not that I know of. I've seen an UML diagram purporting to be "the > Python type herarchy" but it has all of the same flaws your first > attempt exposed, and a lot more time was spent. :-( I haven't seen the UML model you have, but I do know that most of my effort spent toward interfaces didn't attempt to define any of them at all, just the framework for them. There has been a slight amount of effort from what I've seen, spelled out in Python, toward this goal, but not much. I would be interested in seeing that model you refer to. Who showed to you? -Michel From paulp@ActiveState.com Wed Mar 14 02:20:51 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Tue, 13 Mar 2001 18:20:51 -0800 Subject: [Types-sig] Protocols Message-ID: <3AAED583.73BAD76C@ActiveState.com> I agree with Tim that there is no requirement to get the protocols a hundred percent right. If we make them "too heavyweight" then people will just choose to not implement all of the methods or will inherit from UserList to get some for free. Code that depends on those methods will crash just as it does today. If we make them "too lightweight", code will depend on methods not strictly required by the interface and when those methods are missing, it will crash just as it does today. So code still crashes at runtime. That's the cost of a dynamically typed language. BUT we can still help people to avoid passing dictionaries (mappings) for lists (sequences) and vice versa and that is still better than nothing. The minimalist spirit is to lean towards "too lightweight." Therefore I propose that the protocol for mappings require only __getattr__, .keys(), .values() and .items(). The protocol for sequences requires only __getattr__. If someone wants to make a richer set of features on THEIR sequences or mappings, they can define their own types as Tim mentioned. Those can be subtypes of the basics we define. In the future the system will stop guessing at the types of things based on their methods. If they don't have an actual interface declaration saying that they are trying to emulate a mapping or sequence (or whatever), we'll issue a warning that Python doesn't like to guess their intentions. In the meantime, we will guess a little. If we see only __getattr__, we'll presume a sequence. -- Python: Programming the way Guido indented it. - (originated with Skip Montanaro?) From paulp@ActiveState.com Wed Mar 14 02:21:14 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Tue, 13 Mar 2001 18:21:14 -0800 Subject: [Types-sig] File Protocol Proposal Message-ID: <3AAED59A.AB2AB4AA@ActiveState.com> Writefile objects will be required only to have a .write() method. Readfile objects will be required only to have a .read([size]) method. Users will be strongly encouraged to inherit from base classes that define other methods (e.g. readlines) in terms of the required method. e.g. class UserReadFile: def close(): self.closed=true def flush(): pass def isatty(): return false def readline(): while not self.closed and char is not newline: read() def readlines(): read a buncha lines... def read(): raise NotImplementedException -- Python: Programming the way Guido indented it. - (originated with Skip Montanaro?) From paulp@ActiveState.com Wed Mar 14 03:04:50 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Tue, 13 Mar 2001 19:04:50 -0800 Subject: [Types-sig] If I wasn't already... Message-ID: <3AAEDFD2.F74E36FF@ActiveState.com> afraid of parameterized types, a warning I ran into today would convince me: C:\Programs\VISUAL~1\VC98\INCLUDE\xtree(145) : warning C4786: 'std::_Tree,std::allocator>,std::pair,std::allocator > const ,char>,std::map,std::allocator>,char,std::less,std::allocator> >,s td::allocator>::_Kfn,std::less,std::allocator > >,std::allocator>::iterator::operator==' : identifier was truncated to '255' characters in the debug information -- Python: Programming the way Guido indented it. - (originated with Skip Montanaro?) From michel@digicool.com Wed Mar 14 05:53:04 2001 From: michel@digicool.com (Michel Pelletier) Date: Tue, 13 Mar 2001 21:53:04 -0800 (PST) Subject: [Types-sig] Protocols In-Reply-To: <3AAED583.73BAD76C@ActiveState.com> Message-ID: On Tue, 13 Mar 2001, Paul Prescod wrote: > I agree with Tim that there is no requirement to get the protocols a > hundred percent right. If we make them "too heavyweight" then people > will just choose to not implement all of the methods or will inherit > from UserList to get some for free. Code that depends on those methods > will crash just as it does today. This is what I think about what thinking about bringing types to python: to me, protocol means the same thing as interface. Is this off-base? > If we make them "too lightweight", code will depend on methods not > strictly required by the interface and when those methods are missing, > it will crash just as it does today. > > So code still crashes at runtime. That's the cost of a dynamically typed > language. BUT we can still help people to avoid passing dictionaries > (mappings) for lists (sequences) and vice versa and that is still better > than nothing. Agreed. > The minimalist spirit is to lean towards "too lightweight." Therefore I > propose that the protocol for mappings require only __getattr__, > .keys(), .values() and .items(). The protocol for sequences requires > only __getattr__. Really? Defining a sequence as getattr seems to broad to me. To me, something that only implements getattr is an object. Of course, this doesn't work for types, which are objects. I would think a sequence was defined by __len__ and __getitem__. > If someone wants to make a richer set of features on > THEIR sequences or mappings, they can define their own types as Tim > mentioned. Those can be subtypes of the basics we define. > > In the future the system will stop guessing at the types of things based > on their methods. If they don't have an actual interface declaration > saying that they are trying to emulate a mapping or sequence (or > whatever), we'll issue a warning that Python doesn't like to guess their > intentions. > > In the meantime, we will guess a little. If we see only __getattr__, > we'll presume a sequence. We've made a couple of guesses along these lines, should I post them for discussion? -Michel From sverker.is@home.se Wed Mar 14 13:20:22 2001 From: sverker.is@home.se (Sverker Nilsson) Date: Wed, 14 Mar 2001 14:20:22 +0100 Subject: [Types-sig] Interface PEP References: Message-ID: <3AAF7016.7164FDC2@home.se> I was wondering why, or if, the new concept of "interface" should be introduced. There is already the concept of "type" in Python. I thought that concept could/would/should be extended to cover more general interfaces. One would say then, I suppose, instead of interface x... something like typedef x... and the rest of the def would be essentially the same, I think. Instead of the __implements__ special attribute, one would use an attribute name more alluding to the type concept, I might want to call it __type__. The type() builtin would return, as usual, InstanceType if the __type__ special attribute was not defined. Otherwise it would return what __type__ returned. - which would be a user defined type (aka interface) or even a builtin type, if the class wants to claim it emulates a built-in type. (Claiming is one thing of course, really doing it is another - there can never be any guarantee, I suppose.) So.. what would be the reason to have interface another concept than type? Or should they be the same concept? How about the __type__ name for the special attribute, aka __implements__? Regards, Sverker Nilsson From paulp@ActiveState.com Wed Mar 14 13:54:55 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Wed, 14 Mar 2001 05:54:55 -0800 Subject: [Types-sig] Protocols References: Message-ID: <3AAF782F.4F52916A@ActiveState.com> Michel Pelletier wrote: > >... > > This is what I think about what thinking about bringing types to python: > to me, protocol means the same thing as interface. Is this off-base? No, I think that most people use the words interchangably. >... > > The minimalist spirit is to lean towards "too lightweight." Therefore I > > propose that the protocol for mappings require only __getattr__, > > .keys(), .values() and .items(). The protocol for sequences requires > > only __getattr__. > > Really? Defining a sequence as getattr seems to broad to me. To me, > something that only implements getattr is an object. Of course, this > doesn't work for types, which are objects. I would think a sequence was > defined by __len__ and __getitem__. Doh! I meant __getitem__. I'm not sure about __len__ . There are such things as infinite sequences. > We've made a couple of guesses along these lines, should I post them for > discussion? Yes please! -- Python: Programming the way Guido indented it. - (originated with Skip Montanaro?) From paulp@ActiveState.com Wed Mar 14 14:40:58 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Wed, 14 Mar 2001 06:40:58 -0800 Subject: [Types-sig] Sample declarations References: <3AAE220A.DC92ABED@ActiveState.com> <200103131452.JAA32144@cj20424-a.reston1.va.home.com> <3AAE5654.60539291@ActiveState.com> <200103140121.UAA02571@cj20424-a.reston1.va.home.com> Message-ID: <3AAF82FA.9266BDA2@ActiveState.com> Guido van Rossum wrote: > > ... > > No, the situation is approximately as murly as for sequences, mappings > or files. Okay, I'll define Number as the union of complex, integer and float. If anyone wants it to be more general then that they can work out the "general principles" of numerics. >.. > > It's an implementation quirk that this works -- it really shouldn't! > Note that even 5.1 as the buffer size works -- and *that* *really* > shouldn't be allowed. Okay, I'll make the various types non-interchangable for now and then loosen the rules when we have a proper type hierarchy. > What you want is a cast. We have cast functions named after the > concrete types, e.g. int(), float(), long(), etc. I would like to have a single cast operator that takes a type object. We could ask both the object and the type if they knew how to cast back and forth. So if you pass an int or complex where code needs a float, the object would be automatically casted. I'm not going to design a more generalized casting model as part of this effort but I have a sense that if we had such a thing, it would be much easier to ensure consistency of casting everywhere. C extensions would behave the same as operators which would behave the same as Python code... > Can I make a comment on these comments? You seem to be using comments > only when there's something not 100% kosher in your code. An implied > "XXX". :-) I'd appreciate to see more comments that actually explain > what you are doing and why, rather than what you are *not* (yet) > doing! (Most of JPython was once written using this convention -- > while it was clean code, it was very hard to read for the second > developer...) Okay. -- Python: Programming the way Guido indented it. - (originated with Skip Montanaro?) From tim.hochberg@ieee.org Wed Mar 14 16:06:39 2001 From: tim.hochberg@ieee.org (Tim Hochberg) Date: Wed, 14 Mar 2001 09:06:39 -0700 Subject: [Types-sig] File Protocol Proposal References: <3AAED59A.AB2AB4AA@ActiveState.com> Message-ID: <013e01c0aca0$c0bb9e80$87740918@CX781526B> From: "Paul Prescod" > Writefile objects will be required only to have a .write() method. > > Readfile objects will be required only to have a .read([size]) method. This sounds good. For reasons that I'll describe below though it might be better for them to have different names. perhaps BasicWritefile and BasicReadfile. > Users will be strongly encouraged to inherit from base classes that > define other methods (e.g. readlines) in terms of the required method. > e.g. > > class UserReadFile: [A class that implements all file read methods in terms of just read] This seems like an excellent idea! I'll see if I can come up with an equivalent for readable and writable sequences. Here's a couple of obeservations in no particular order: Again I don't like the name UserReadFile. I think it suggests a false parallel with UserList and UserDict. Those exist in order to subclass the builtin list and dictionary classes, the above class has an entirely different meaning. If we are going to have a baseclass that users will be encouraged to inherit from when defining there own file-like object, it should be possible to require it in a type declararion. I'd suggest we use ReadFile (ReadableFile, FileReader?) for the wide interface and BasicReadFile for the narrow interface. It's tempting to overload the above class to allow it act as a wrapper for a basic readfile. So that I can use UserReadFile(aBasicReadFile) to get a full featured ReadFile. I think that would just be a matter of adding an __init__ method. I'm not positive this would be a good idea, but I believe that this is what would need to be added: def __init__(self, wrapped=None): if wrapped is not None: self.read = wrapped.read One an unrelated note, I didn't like the idea of using & and | when someone else mentioned, since it seemed to be leading down the road to static typing (a road to nowhere IOW). However, if typecheck objects inherited from a common base class it would be pretty easy to implement in terms of magic methods. This would mean that "TypeA | TypeB | TypeC" would mean then same as what I originally called CompositeOf(TypeA, TypeB, TypeC). -tim From michel@digicool.com Wed Mar 14 17:33:46 2001 From: michel@digicool.com (Michel Pelletier) Date: Wed, 14 Mar 2001 09:33:46 -0800 (PST) Subject: [Types-sig] Interface PEP In-Reply-To: <3AAF7016.7164FDC2@home.se> Message-ID: On Wed, 14 Mar 2001, Sverker Nilsson wrote: > I was wondering why, or if, the new concept of "interface" should be > introduced. There is already the concept of "type" in Python. I > thought that concept could/would/should be extended to cover more > general interfaces. > > One would say then, I suppose, instead of > > interface x... > > something like > > typedef x... > > and the rest of the def would be essentially the same, I think. > Instead of the __implements__ special attribute, one would use > an attribute name more alluding to the type concept, I might want > to call it __type__. > > The type() builtin would return, as usual, InstanceType if the > __type__ special attribute was not defined. Otherwise it would return > what __type__ returned. - which would be a user defined type (aka > interface) or even a builtin type, if the class wants to claim it > emulates a built-in type. > > (Claiming is one thing of course, really doing it is another - > there can never be any guarantee, I suppose.) > > So.. what would be the reason to have interface another concept than > type? > > Or should they be the same concept? They are different concepts. Read the section in the PEP on "Classes and Interfaces". One class, or "type" of object, can implement multiple interfaces, and a "type" consisting of an many mixed in classes can be used to implement just one interface. Many different objects of different types many also implement the *same* interface. For example, lots of different types of objects can be a sequence or a mapping. "sequence" and "mapping" are not types like lists and dictionaries, they are just protocol descriptions, or interfaces. In Zope, we may have a RelationalUserFolder, and a FileUserFolder, both completely different implementations and different types, but they both implement the UserFolderInterface. In many ways, interfaces achieve a lot of what a stronger typing system in python strives to achieve. This doesn't mean they can't co-exist, and I posit any idea type system in python would be intimately involved with the interfaces (often refered to as "protocols") of objects. -Michel From barry@digicool.com Wed Mar 14 17:50:37 2001 From: barry@digicool.com (Barry A. Warsaw) Date: Wed, 14 Mar 2001 12:50:37 -0500 Subject: [Types-sig] File Protocol Proposal References: <3AAED59A.AB2AB4AA@ActiveState.com> Message-ID: <15023.44909.286985.224674@anthem.wooz.org> | def isatty(): | return false Two thoughts of the day... We may need to add a boolean type to Python. How would you describe today a method that takes a true/false flag? There's a lot of variety in how you can spell true and false today. Writing a boolean class isn't hard -- I've done it at least twice . Writing a boolean type was more difficult because of the old comparison rules; I haven't tried since richcomps have been added so the situation might be better now. There's also the common situation of accepting either None or a specific object. I.e. "this method takes None or a Foobar instance". And what about a method that might accept any of None, a string (filename), or a "file-like" object? This is all very common in Python programs. Finally, another thought occurred to me, which might be based on a mis-remembering of what ObjC does (I don't have any of my old ObjC books laying around, and it's been a while). Say a method takes an object that must have a read([size]) method. That's all you care about. Yes, you could define an interface that describes this, but what if someone passes in an object that conforms to a full blown "ReadableFile" interface -- i.e. it provides read([size]), readline(), readlines(), and maybe seek()? Let's say those two interfaces are defined in unrelated and separately authored libraries, but I'm writing code that combines the two. Library B gives me a factory that produces ReadableFiles and I'd like to pass those to library B's method that specifies a MustHaveRead interface. Am I screwed? Or do I have to play convoluted games of multiple inheritance, mixins, interface mishmashing, etc? To a newbie (and maybe a not-so-newbie), it ought to be perfectly legal. ReadableFile's interface is a superset of MustHaveRead so it should be fine, but because the interface objects aren't explicitly related, you're hosed. What you want (maybe) is interface conformance based on a runtime check of the interface specifications. Maybe a kind of interface arithmetic? IIRC, this was one of the ObjC uses of categories. A category is just a collection of methods, so if you have a category that contained only read([size]), and you declared a parameter to be of the "type" of that category, you could pass in any object that had such a method. Well, back to peripherally following this list. :) -Barry From qrczak@knm.org.pl Wed Mar 14 17:24:33 2001 From: qrczak@knm.org.pl (Marcin 'Qrczak' Kowalczyk) Date: 14 Mar 2001 17:24:33 GMT Subject: [Types-sig] If I wasn't already... References: <3AAEDFD2.F74E36FF@ActiveState.com> Message-ID: Tue, 13 Mar 2001 19:04:50 -0800, Paul Prescod pis= ze: > afraid of parameterized types, a warning I ran into today would > convince me: I once got a better one: mico-c++ -Wall -I. -c serwer.cc -o serwer.o serwer.cc: In method `void Firma_impl::usun(const char *)': serwer.cc:92: `struct __rb_tree_iterator,__default_alloc_template >,Katalog *>,const= pair,__default_alloc_te= mplate >,Katalog *> &,const pair,__default_alloc_template >,Katalog *> *>' has no= member named `second' serwer.cc:93: no matching function for call to `map,__default_alloc_template >,Katalog *,less,__default_alloc_template > >,__default_alloc_template >::erase (__rb_tree_iterator,__default_alloc_template= >,Katalog *>,const pair,__default_alloc_template >,Katalog *> &,const pair,__default_alloc_template >,Katalog *> *> &)' /usr/include/g++/stl_map.h:156: candidates are: map,__default_alloc_template >,Katalog *,less,__default_alloc_template > >,__default_alloc_template >::erase,__default_alloc_template > >, alloc>(__rb_tree_iterator,__default_alloc_template >,Katalog *>,pair,__default_alloc_template= >,Katalog *> &,pair,__d= efault_alloc_template >,Katalog *> *>) /usr/include/g++/stl_map.h:157: map,__default_alloc_template >,Katalog *,less,__default_alloc_template > >,__default_alloc_template >::erase,__default_alloc_template > >, alloc>(const string &) /usr/include/g++/stl_map.h:158: map,__default_alloc_template >,Katalog *,less,__default_alloc_template > >,__default_alloc_template >::erase,__default_alloc_template > >, alloc>(__rb_tree_iterator,__default_alloc_template >,Katalog *>,pair,__default_alloc_template= >,Katalog *> &,pair,__d= efault_alloc_template >,Katalog *> *>, __rb_tree_iterator,__default_alloc_template >,Katalog *>,pair= ,__default_alloc_template >,Katalog *> &,pair, __default_alloc_template >,Katalog *> *>) make: *** [serwer.o] Error 1 But please don't judge static typing from bad examples like C++ or Java! --=20 __("< Marcin Kowalczyk * qrczak@knm.org.pl http://qrczak.ids.net.pl/ \__/ ^^ SYGNATURA ZAST=CAPCZA QRCZAK From paulp@ActiveState.com Wed Mar 14 18:08:25 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Wed, 14 Mar 2001 10:08:25 -0800 Subject: [Types-sig] Proposed type declaration syntax Message-ID: <3AAFB399.3B7F1525@ActiveState.com> While we are experimenting, I want to use a type declaration syntax that does not interfere with standard Python syntax. I'd also like it to be discoverable both at runtime (for ERR) and compile time (for DOC). def func(module, integer, func, stream, tuple, string, unknown): __types__ = {"module: A module to foobar": types.ModuleType, "integer: An integer to baz": Int, "func: A callable object": Callable, "stream: A readable stream": ReadStream, "tuple: A sequence": Sequence, "string: A string": String, "unknown: Some weird type": None, "__returns__: File handle no":Int} __asserttypes() ... some long calculation... return __assertrc(42) Note that you can *declare types* for purposes of DOC without *asserting types* for purposes of ERR. Those who like DOC but hate ERR have a recourse. I'll post the code that implements this later today and then start working on a PEP for the Zwiki and sourceforge. I could use function attributes for this but I chose not to for the following reasons (which I will document in the PEP): * function attributes are implied to be sort of more dynamic in that they can be assigned from anywhere * function attributes must go AFTER the code which is a big problem for long functions. If anything, I would be more inclined to move the type declaration BEFORE the function definition, not after the suite I also considered a more verbose but explicit format like assertTypes(__types__, locals()) but the __assertrc got too ugly. -- Python: Programming the way Guido indented it. - (originated with Skip Montanaro?) From michel@digicool.com Wed Mar 14 18:11:32 2001 From: michel@digicool.com (Michel Pelletier) Date: Wed, 14 Mar 2001 10:11:32 -0800 (PST) Subject: [Types-sig] Protocols In-Reply-To: <3AAF782F.4F52916A@ActiveState.com> Message-ID: (Warning, sloppy UML linked) On Wed, 14 Mar 2001, Paul Prescod wrote: > Michel Pelletier wrote: > > > > This is what I think about what thinking about bringing types to python: > > to me, protocol means the same thing as interface. Is this off-base? > > No, I think that most people use the words interchangably. Good. > > > The minimalist spirit is to lean towards "too lightweight." Therefore I > > > propose that the protocol for mappings require only __getattr__, > > > .keys(), .values() and .items(). The protocol for sequences requires > > > only __getattr__. > > > > Really? Defining a sequence as getattr seems to broad to me. To me, > > something that only implements getattr is an object. Of course, this > > doesn't work for types, which are objects. I would think a sequence was > > defined by __len__ and __getitem__. > > Doh! I meant __getitem__. I'm not sure about __len__ . There are such > things as infinite sequences. Yes, that makes sense. > > We've made a couple of guesses along these lines, should I post them for > > discussion? > > Yes please! Okay, here are some experimental concepts we've worked with. Keep in mind all of you, that these are just the way things look like to *us*. There may be better names for these interfaces. We may have factored them different than you would. etc. etc. This is the kind of thing the BDFL would pronounce on if there were many different opinions. For this email, I will use my notional "interface" syntax, but our prototypes use the Zope reference implementation: interface Mutable: """Mutable objects """ interface Comparable: """Objects that can be tested for equality """ interface Orderable(Comparable): """Objects that can be compared with < > == <= >= !=""" interface Hashable: """Objects that support hash""" interface HashKey(Comparable, Hashable): """Objects that are immutable with respect to state that influences comparisons or hash values""" interface Mapping: "anything supporting __getitem__" interface Sized: "anything supporting __len" interface MutableMapping(Mutable): "Has __setitem__ and __delitem__" interface Sequence(Mapping): "Keys must be integers in a sequence starting at 0." interface Sequential(Sequence): "Keys must be used in order" I have captured this in a UML model to make the concept more clear in relation to three simple types: tuple, list, and dictionary (for bonus points, you can try and guess which of these interfaces these three types implement before you look at the model). http://www.zope.org/Members/michel/PythonUML.png -Michel From tim.hochberg@ieee.org Wed Mar 14 18:31:26 2001 From: tim.hochberg@ieee.org (Tim Hochberg) Date: Wed, 14 Mar 2001 11:31:26 -0700 Subject: [Types-sig] File Protocol Proposal References: <3AAED59A.AB2AB4AA@ActiveState.com> <15023.44909.286985.224674@anthem.wooz.org> Message-ID: <025d01c0acb4$faef6e60$87740918@CX781526B> ----- Original Message ----- > > | def isatty(): > | return false > > Two thoughts of the day... > > We may need to add a boolean type to Python. How would you describe > today a method that takes a true/false flag? There's a lot of variety > in how you can spell true and false today. > > Writing a boolean class isn't hard -- I've done it at least twice > . Writing a boolean type was more difficult because of the old > comparison rules; I haven't tried since richcomps have been added so > the situation might be better now. It's possible that this has changed, but according so section 2.1.1 of the Library Reference, everything is a boolean type! Or am I missing something. > There's also the common situation of accepting either None or a > specific object. I.e. "this method takes None or a Foobar instance". > And what about a method that might accept any of None, a string > (filename), or a "file-like" object? This is all very common in > Python programs. The simple way is to just use union type. I called this composite in my orginal proposal, but that may be bad terminology. Anyway, the following should work: def weeble(wobble : UnionOf(NoneType, String)): # .... If we forced typecheck objects to all inherit from a common base class, it would also be pretty easy to write this as: def weeble(wobble : NoneType | String): # .... By suitably overloading __or__ in the base class. One would probably also overload __and__ while one was at it. > Finally, another thought occurred to me, which might be based on a > mis-remembering of what ObjC does (I don't have any of my old ObjC > books laying around, and it's been a while). > > Say a method takes an object that must have a read([size]) method. > That's all you care about. Yes, you could define an interface that > describes this, but what if someone passes in an object that conforms > to a full blown "ReadableFile" interface -- i.e. it provides > read([size]), readline(), readlines(), and maybe seek()? Let's say > those two interfaces are defined in unrelated and separately authored > libraries, but I'm writing code that combines the two. Library B > gives me a factory that produces ReadableFiles and I'd like to pass > those to library B's method that specifies a MustHaveRead interface. > > Am I screwed? No. >Or do I have to play convoluted games of multiple > inheritance, mixins, interface mishmashing, etc? To a newbie (and > maybe a not-so-newbie), it ought to be perfectly legal. > ReadableFile's interface is a superset of MustHaveRead so it should be > fine, but because the interface objects aren't explicitly related, > you're hosed. No. Unless I'm misunderstanding you. All of this should "just work". The __implementedby__ method of your narrow interface should just be checking whether a given object supports read. It doesn't care if another interface was defined in another file. > What you want (maybe) is interface conformance based on a runtime > check of the interface specifications. Maybe a kind of interface > arithmetic? That's basically the plan. Well except for the arithmatic part. >IIRC, this was one of the ObjC uses of categories. A > category is just a collection of methods, so if you have a category > that contained only read([size]), and you declared a parameter to be > of the "type" of that category, you could pass in any object that had > such a method. That's pretty close to what we're trying. The hard part is coming up with a set of basic categories that captures much of existing usage without being ludicrously large. Users can of course write there own typecheck classes, but you'd like the common cases to be captured by the supplied typecheckers. -tim From sverker.is@home.se Wed Mar 14 19:53:05 2001 From: sverker.is@home.se (Sverker Nilsson) Date: Wed, 14 Mar 2001 20:53:05 +0100 Subject: [Types-sig] Interface PEP References: Message-ID: <3AAFCBDF.39A0735D@home.se> Michel Pelletier wrote: > > On Wed, 14 Mar 2001, Sverker Nilsson wrote: > > > I was wondering why, or if, the new concept of "interface" should be > > introduced. There is already the concept of "type" in Python. I > > thought that concept could/would/should be extended to cover more > > general interfaces. > > > > One would say then, I suppose, instead of > > > > interface x... > > > > something like > > > > typedef x... > > > > and the rest of the def would be essentially the same, I think. > > Instead of the __implements__ special attribute, one would use > > an attribute name more alluding to the type concept, I might want > > to call it __type__. > > > > The type() builtin would return, as usual, InstanceType if the > > __type__ special attribute was not defined. Otherwise it would return > > what __type__ returned. - which would be a user defined type (aka > > interface) or even a builtin type, if the class wants to claim it > > emulates a built-in type. > > > > (Claiming is one thing of course, really doing it is another - > > there can never be any guarantee, I suppose.) > > > > So.. what would be the reason to have interface another concept than > > type? > > > > Or should they be the same concept? > > They are different concepts. Read the section in the PEP on "Classes and Ok, I read it again closer now. Well, what I can agree with is that the interfaces are _descriptions_: more for the human reader, than for (dynamic or static) type-checking. I could consider the type of an object to be something different than the interface, but then _also_ different from the class, contrary to what you imply below. Maybe a useful combination of the type and interface objects would be that the type _contained_ an (optional) interface description. This would not need to be used by the typechecker, which would compare the types directly. > Interfaces". One class, or "type" of object, can implement multiple So you identify the object's class with its type... why? > interfaces, and a "type" consisting of an many mixed in classes can be > used to implement just one interface. > > Many different objects of different types many also implement the *same* > interface. For example, lots of different types of objects can be a > sequence or a mapping. Well I'd say "lots of different _classes_ of objects.." Not types of - that would again prematurely fix what we mean with the type of an object. > "sequence" and "mapping" are not types like lists > and dictionaries, they are just protocol descriptions, or interfaces. I'd say SequenceType or MappingType should be types just as well as ListType. Why make them different kinds of things? I'd say ListType is a _subtype_ of SequenceType. With this I mean: If we specify a function f that takes a parameter a of type x: def f(a:x):... and call it: f(y) it is required that the type of y is a subtype of x. So if x was 'SequenceType', it would be possible to call it if type(y) == ListType, TupleType, SequenceType, or any other type that was a subtype of SequenceType. This could be checked (automatically) with for example: assert subtype(type(y), x) or maybe assert x.is_supertype_of(type(y)) Using interfaces as types, I guess one could do it with isInstance on the interface object. (Or maybe one wants something more flexible.) > In Zope, we may have a RelationalUserFolder, and a FileUserFolder, both > completely different implementations and different types, but they both > implement the UserFolderInterface. > > In many ways, interfaces achieve a lot of what a stronger typing system in > python strives to achieve. This doesn't mean they can't co-exist, and I > posit any idea type system in python would be intimately involved with the What does posit mean? (Not in my dict.) Like 'appreciate'? I suppose so in the following! > interfaces (often refered to as "protocols") of objects. Well am I right to understand that interfaces are not meant for typechecking, but mostly for a kind of description? Then I think they would still be useful, maybe as a field in the actual type objects. Here is my tongue-in-cheek micro-suggestion then :-) Objects could have a __type__ field. The builtin function type() returns the __type__ field if defined. Type objects can be user-defined or builtin. Type objects should implement at least a subtype or supertype method. They can also have an interface field for the interface description. Sverker From michel@digicool.com Wed Mar 14 20:13:37 2001 From: michel@digicool.com (Michel Pelletier) Date: Wed, 14 Mar 2001 12:13:37 -0800 (PST) Subject: [Types-sig] Interface PEP In-Reply-To: <3AAFCBDF.39A0735D@home.se> Message-ID: On Wed, 14 Mar 2001, Sverker Nilsson wrote: > Ok, I read it again closer now. Well, what I can agree with is that > the interfaces are _descriptions_: more for the human reader, than for > (dynamic or static) type-checking. Their role in type checking is not determined, and the concept is (possibly) out of scope of the PEP. They can, however, be used for run-time checking in many flexible ways, including, I belive, type checking. > I could consider the type of an object to be something different > than the interface, but then _also_ different from the class, > contrary to what you imply below. Yeah I was just mixed up. I wasn't thinking along the lines of the very specific definition of type and looking toward the future where types and classes become one. Maybe I missunderstand the core of the type/class dichotomy problem. > Maybe a useful combination of the type and interface objects would be > that the type _contained_ an (optional) interface description. Another useful combination is the exact opposite, the interface describes the expected types of the implementation. > > Many different objects of different types many also implement the *same* > > interface. For example, lots of different types of objects can be a > > sequence or a mapping. > > "sequence" and "mapping" are not types like lists > > and dictionaries, they are just protocol descriptions, or interfaces. > > I'd say SequenceType or MappingType should be types just as well as > ListType. Why make them different kinds of things? Well, in my world, sequence and mapping are interfaces, not types. This is not only a slippery slope, but a hairy issue indeed. > I'd say ListType is a _subtype_ of SequenceType. With this I mean: I'd say that lists implement the sequence interface. > Using interfaces as types, I guess one could do it with isInstance on > the interface object. (Or maybe one wants something more flexible.) You would ask the object if it impliments the interface you're looking for. > > In Zope, we may have a RelationalUserFolder, and a FileUserFolder, both > > completely different implementations and different types, but they both > > implement the UserFolderInterface. > > > > In many ways, interfaces achieve a lot of what a stronger typing system in > > python strives to achieve. This doesn't mean they can't co-exist, and I > > posit any idea type system in python would be intimately involved with the > > What does posit mean? (Not in my dict.) Like 'appreciate'? I suppose > so in the following! Oh it may be slang, it means "to have a position" or "assert". > > interfaces (often refered to as "protocols") of objects. > > Well am I right to understand that interfaces are not meant for > typechecking, but mostly for a kind of description? They could concievable do type checking. > Then I think they would still be useful, maybe as a field in the > actual type objects. Or, as I said, interfaces could, vice versa, assert their elements must be of certain types. -Michel From paulp@ActiveState.com Wed Mar 14 20:37:11 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Wed, 14 Mar 2001 12:37:11 -0800 Subject: [Types-sig] Interface PEP References: Message-ID: <3AAFD677.7037E987@ActiveState.com> It's important to realize that both sides of the current discussion are working on interfaces. Michel's interfaces are defined declaratively in a class-like syntax. Mine are basically functional interfaces (in fact they could really be functions instead of classes). Your interface objects have an implicitly generated implementedBy method. Mine have an explicit method. -- Python: Programming the way Guido indented it. - (originated with Skip Montanaro?) From qrczak@knm.org.pl Wed Mar 14 20:36:07 2001 From: qrczak@knm.org.pl (Marcin 'Qrczak' Kowalczyk) Date: 14 Mar 2001 20:36:07 GMT Subject: [Types-sig] Interface PEP References: <3AAF7016.7164FDC2@home.se> Message-ID: Wed, 14 Mar 2001 14:20:22 +0100, Sverker Nilsson pis= ze: > The type() builtin would return, as usual, InstanceType if the > __type__ special attribute was not defined. Otherwise it would return > what __type__ returned. - which would be a user defined type (aka > interface) or even a builtin type, if the class wants to claim it > emulates a built-in type. Warning, warning! This is a wrong way. There is no concept of asking an object which interface it implements. There is no "the" interface it implements. It's not even a set of interfaces, because the object doesn't know them in advance. Interfaces can be defined after objects conforming to them are created. This is a thing which languages like Java, Eiffel, OO Pascal extensions, C#, and the OO part of C++ got wrong. They require the object to know interfaces it implements. This limits the flexibility of interfaces. This is a reason why languages like Python are dynamically typed. Interfaces can be designed well in statically typed languages. Haskell, Clean, Mercury, OCaml, the template part of C++ (poorly), and an extension of g++ called "signatures" (to be dropped or already dropped) - don't require the object to know its interfaces in advance. Some of these (especially the Haskell / Clean / Mercury solution) also don't require to list a single interface when specifying what argument a function accepts. This is done in various ways: * Haskell, Clean and Mercury have a powerful system of what they call "classes". They should be called "interfaces". A separate declaration asserts that a type conforms to an interface, by providing implementations of methods the interface requires. Methods don't have a distinguished object they act on but look like plain functions - e.g. (+) is fully symmetric. * OCaml has an OO class system where interfaces are implicitly created by listing methods with their types (of course you can bind them to names). An object conforms to an interface if it has these methods with compatible types. * C++ doesn't have interfaces formally in the language. A template can be instantiated on any type. If the type provides the right functionality (by having appropriately named functions overloaded), the program works. If it doesn't provide it, the program may or may not compile, may or may not work properly, and it may depend on the version of the library (which parts of the interface it uses in which way). Interfaces are only roughly defined and are only described in human languages. * C++ signatures are similar to OCaml in the concept, but with much worse quality of design and implementation. It would be silly to lead Python to the wrong direction, exemplified by Java and the OO part of C++. Given how Python looks today, it will not be accepted, because having to design all interfaces from the beginning is too constraining. The failure will be attributed to static typing. So what to do? The Haskell / Clean / Mercury solution is probably too far from Python. The template part of C++ is approximately what we have today, only without static typing. We can aim at the OCaml / C++ signatures direction. So an interface is defined by listing method names and requirements. It can be given a name, but a class doesn't need to list interface names in order to conform to them. Of course objects don't know their interfaces, similarly as they don't know names of variables they are bound to. We can also aim at the way I described as wrong, provided that it is sufficiently dynamic. So an object or a class keeps the list of interfaces it implements, and they can be added on the fly later. This way is less formal if conforming to a protocol is not checked but declared. We can try a combination of both. Or something completely different. --=20 __("< Marcin Kowalczyk * qrczak@knm.org.pl http://qrczak.ids.net.pl/ \__/ ^^ SYGNATURA ZAST=CAPCZA QRCZAK From michel@digicool.com Wed Mar 14 20:39:17 2001 From: michel@digicool.com (Michel Pelletier) Date: Wed, 14 Mar 2001 12:39:17 -0800 (PST) Subject: [Types-sig] Interface PEP In-Reply-To: Message-ID: On Wed, 14 Mar 2001, Michel Pelletier wrote: > Another useful combination is the exact opposite, the interface describes > the expected types of the implementation. I added a section to the PEP 'Type Checking' near the bottom, that describes more what I'm thinking about along these lines. http://www.zope.org/Members/michel/InterfacesPEP/PEP.txt -Michel From michel@digicool.com Wed Mar 14 20:47:44 2001 From: michel@digicool.com (Michel Pelletier) Date: Wed, 14 Mar 2001 12:47:44 -0800 (PST) Subject: [Types-sig] Interface PEP In-Reply-To: <3AAFD677.7037E987@ActiveState.com> Message-ID: On Wed, 14 Mar 2001, Paul Prescod wrote: > It's important to realize that both sides of the current discussion are > working on interfaces. Yup. > Michel's interfaces are defined declaratively in > a class-like syntax. Well, the syntax is a secondary issue, I'm concerned more with the model. I think what you are saying is that my proposal specifies that specification and implementation must be seperate. Or maybe you are saying what you said, in that case, you lost me. > Mine are basically functional interfaces (in fact > they could really be functions instead of classes). You lost me there too. ;) > Your interface objects have an implicitly generated implementedBy > method. Mine have an explicit method. I don't get this either, can you be more explicit about your explicitness, also by "your" I assume you mean me? Above you were referring to me in the third person, so I'm lost yet again. -Michel From michel@digicool.com Wed Mar 14 20:56:30 2001 From: michel@digicool.com (Michel Pelletier) Date: Wed, 14 Mar 2001 12:56:30 -0800 (PST) Subject: [Types-sig] Interface PEP In-Reply-To: Message-ID: On Wed, 14 Mar 2001, Michel Pelletier wrote: > > > On Wed, 14 Mar 2001, Paul Prescod wrote: > > > It's important to realize that both sides of the current discussion are > > working on interfaces. > > Yup. > > > Michel's interfaces are defined declaratively in > > a class-like syntax. > > Well, the syntax is a secondary issue, I'm concerned more with the model. I added some more stuff to the PEP to help distinguish between the model and the syntax. http://www.zope.org/Members/michel/InterfacesPEP/PEP.txt -Michel From paulp@ActiveState.com Wed Mar 14 21:06:23 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Wed, 14 Mar 2001 13:06:23 -0800 Subject: [Types-sig] Interface PEP References: Message-ID: <3AAFDD4F.C40686C@ActiveState.com> Michel Pelletier wrote: > >... > > Well, the syntax is a secondary issue, I'm concerned more with the model. > I think what you are saying is that my proposal specifies that > specification and implementation must be seperate. Or maybe you are > saying what you said, in that case, you lost me. I think I was saying what I said. :) Your syntax: interface ReadableFile: def read(size): pass >>> print dir(ReadableFile) [...., implementedBy, ...] >>> class MyReadFile implements ReadFile: ... ... My syntax: >>> class _ReadableFile: ... def __implementedby___(self, obj): ... return hasattr(obj, read) ... ReadFile = _ReadableFile() >>> class MyReadFile: ... __implements__=[ReadFile] > > Mine are basically functional interfaces (in fact > > they could really be functions instead of classes). > > You lost me there too. ;) The operative part of my interfaces is the __implementedby__ method. What you say in a bunch of method declarations, I say in __implementedby__ > > Your interface objects have an implicitly generated implementedBy > > method. Mine have an explicit method. > > I don't get this either, can you be more explicit about your explicitness, > also by "your" I assume you mean me? Above you were referring to me in > the third person, so I'm lost yet again. You create objects that don't LOOK like they have an implementedBy method, but they really do (implicit). I require the creation of an explicit __implementedby__ method. The interface syntax version is more compact and readable for complicated interfaces. The functional version is more flexible. They will probably work together well as alternate syntaxes. -- Python: Programming the way Guido indented it. - (originated with Skip Montanaro?) From michel@digicool.com Wed Mar 14 21:20:41 2001 From: michel@digicool.com (Michel Pelletier) Date: Wed, 14 Mar 2001 13:20:41 -0800 (PST) Subject: [Types-sig] Interface PEP In-Reply-To: <3AAFDD4F.C40686C@ActiveState.com> Message-ID: On Wed, 14 Mar 2001, Paul Prescod wrote: > The interface syntax version is more compact and readable for > complicated interfaces. The functional version is more flexible. They > will probably work together well as alternate syntaxes. Ah, much clearer now. Looking forward to reading your proposal, maybe I can steal some ideas! ;) -Michel From michel@digicool.com Wed Mar 14 21:25:06 2001 From: michel@digicool.com (Michel Pelletier) Date: Wed, 14 Mar 2001 13:25:06 -0800 (PST) Subject: [Types-sig] Interface PEP In-Reply-To: <3AAFDD4F.C40686C@ActiveState.com> Message-ID: On Wed, 14 Mar 2001, Paul Prescod wrote: > > interface ReadableFile: > def read(size): > pass There's a little nugget of gold here, BTW. In the PEP, I had the first argument of interface method's being 'self', like the class syntax. But self is very much an implementation issue, not a specification issue. Looking at what you have here convinced me of that. I've removed the 'self's from the PEP. Thanks, -Michel From qrczak@knm.org.pl Wed Mar 14 21:37:34 2001 From: qrczak@knm.org.pl (Marcin 'Qrczak' Kowalczyk) Date: 14 Mar 2001 21:37:34 GMT Subject: [Types-sig] Interface PEP References: <3AAFCBDF.39A0735D@home.se> Message-ID: Wed, 14 Mar 2001 20:53:05 +0100, Sverker Nilsson pis= ze: > Well, what I can agree with is that the interfaces are _descriptions_: > more for the human reader, than for (dynamic or static) type-checking. They are both for humans and for the interpreter / compiler. > I could consider the type of an object to be something different > than the interface, but then _also_ different from the class, > contrary to what you imply below. There are always problems with underspecified terminology. Especially the area of types, classes, interfaces, subtyping / subclassing / inheritance, polymorphism is really confusing... The same terms are used in the context of different systems or languages with different meaning. Easy to be confused or misunderstood. Python uses terms "type" and "class". There are several builtin types. Object created as class instances all have the same type (InstanceType), and can be further distinguished by their classes. Many people agree that it would be cleaner to unify these concepts in future. C++ uses terms "type" and "class". There is a similarity to Python's usage, but in C++ there is no single InstanceType: some types are classes, some are not classes. Class templates aren't types yet. Eiffel uses terms "type" and "class". Class is the basic unit of modularity (how an object is defined), type describes an interface (what do we expect from an object). Each non-parametrized class yields a type. Each parametrized class may yield a type when applied to appropriate arguments. Haskell and Clean use terms "type" and "class". Type defines an implementation and functions often require a concrete type. Class defines an interface. Types may conform to classes. Functions may also require that the type of their argument conforms to some classes. This is quite backwards compared to Eiffel and OO theoretic terminology. Parametrized types are unified with types, i.e. types of concrete objects are special cases of parametrized types in general. OCaml uses terms "type" and "class". Type defines an implementation and functions often require a concrete type. The OO subsystem introduces classes (templates of objects with method definitions) and class types (interfaces of objects with method signatures). A class type is a type. A class is something between an object and a type. I hope I got it right - I don't have an experience with OCaml's OO subsystem. We have plenty of definitions to choose from :-) > > Interfaces". One class, or "type" of object, can implement multiple >=20 > So you identify the object's class with its type... why? In Python they have the same purpose. A type, combned with a class (in case of InstanceType), describe the implementation of an object. Well, partially: object attributes can be different for each object. Interfaces of objects are not described formally in Python nor checked. One of current goals is to change this. > Well I'd say "lots of different _classes_ of objects.." > Not types of - that would again prematurely fix what we mean > with the type of an object. I assume that we use the term "type" as currently defined in Python, and don't change that. > I'd say SequenceType or MappingType should be types just as well > as ListType. Why make them different kinds of things? ListType describes an implementation, i.e. how an object was constructed. There is a particular object layout and particular methods. SequenceType describes an interface, i.e. how an object can be used. An interace can include things that "len(o) is defined and means blah blah" - it doesn't matter how len is dispatched. > I'd say ListType is a _subtype_ of SequenceType. A basic misunderstanding of some languages is confusing subtyping (the ability to use one type if another is required, i.e. a relation on interfaces) and inheritance (basing a class definition on another class, i.e. reusing the implementation). Eiffel has problems because of this. It requires checks of the whole assembled system to see if a subclass yields a subtype or not. C++ and Java have lack of expressiveness because of this. They require some casts because a method signature can't change in an inherited class. I don't say that you claim that subtyping is the same as inheritance. But you try to unify types / classes with interfaces, which doesn't work well. A type / class may generate an interface requirement: namely that an object is of this class or a subclass (the "or a subclass" part is very problematic). An interface may generate a type / class definition: we may want to hold an arbitrary object conforming to an interface. They are sometimes blurred, but they are definitely different concepts. We can have two interfaces (e.g. "comparable" and "convertible to a string"). One object conforms to the first but not to the second, another conforms to the second but not to the first. This situation cannot be described by locating the object's type in the interface tree and checking its supertypes. It's not a tree. --=20 __("< Marcin Kowalczyk * qrczak@knm.org.pl http://qrczak.ids.net.pl/ \__/ ^^ SYGNATURA ZAST=CAPCZA QRCZAK From qrczak@knm.org.pl Wed Mar 14 21:47:27 2001 From: qrczak@knm.org.pl (Marcin 'Qrczak' Kowalczyk) Date: 14 Mar 2001 21:47:27 GMT Subject: [Types-sig] File Protocol Proposal References: <3AAED59A.AB2AB4AA@ActiveState.com> <15023.44909.286985.224674@anthem.wooz.org> Message-ID: Wed, 14 Mar 2001 12:50:37 -0500, Barry A. Warsaw pis= ze: > We may need to add a boolean type to Python. I agree. But I'm afraid that if Guido wanted it, he would add it earlier. > Say a method takes an object that must have a read([size]) method. > That's all you care about. Yes, you could define an interface > that describes this, but what if someone passes in an object that > conforms to a full blown "ReadableFile" interface -- i.e. it provides > read([size]), readline(), readlines(), and maybe seek()? Of course the system must be designed to make this work. > Let's say those two interfaces are defined in unrelated and > separately authored libraries, but I'm writing code that combines > the two. Library B gives me a factory that produces ReadableFiles > and I'd like to pass those to library B's method that specifies a > MustHaveRead interface. >=20 > Am I screwed? Depending on whether we do it right :-) Other languages I know solve it either: - by declaring that ReadableFiles conform to the MustHaveRead interface; this can be done *outside* to the definition of ReadableFiles, because MustHaveRead requirements can be fulfilled basing on the public interface of ReadableFiles; or - by using "name equivalence" instead of "object identity" for determining if both 'read' methods are really the same thing; this is how it works in Python and Smalltalk, although without static checking: protocols are defined by names of methods and are not declared. > ReadableFile's interface is a superset of MustHaveRead so it should > be fine, but because the interface objects aren't explicitly related, > you're hosed. Indeed. The system must not require ReadableFile to know about MustHaveRead. Let's not repeat Java's mistakes. --=20 __("< Marcin Kowalczyk * qrczak@knm.org.pl http://qrczak.ids.net.pl/ \__/ ^^ SYGNATURA ZAST=CAPCZA QRCZAK From tim.hochberg@ieee.org Wed Mar 14 23:10:56 2001 From: tim.hochberg@ieee.org (Tim Hochberg) Date: Wed, 14 Mar 2001 16:10:56 -0700 Subject: [Types-sig] Another cut at Sequence declarations References: <3AAE220A.DC92ABED@ActiveState.com> Message-ID: <036401c0acdc$0617a790$87740918@CX781526B> This is a multi-part message in MIME format. ------=_NextPart_000_0361_01C0ACA1.59A4F970 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Attached is my attempt to write a set typecheck classes for classes. So far I've only tried to deal with immutable sequence types; mutable ones are on the agenda, but I've used up my allotment of types-sig time for the day. Here's the executive summary, for the rest, see the code: I defined three protocols: Iterable, BasicSequence and Sequence. Iterable is something you can iterate on. Currently that means it supports getitem, but this will be expanded to include whatever the iter-sig comes up with. BasicSequence supports getitem and len. This is the minimum that you need to construct an immutable sequence (e.g., a tuple). Sequence supports the same set of operations as a tuple. So, Sequence extends BasicSequence which in turn extends Iterable. I've also included a base class for users that want to create their own Sequences. This class is called SequenceBase. Only getitem and len must be overriden to get a working Sequence. The combination of the names BasicSequence and SequenceBase is probably bad, so at least one of them should be changed. Finally, there is a class SequenceWrapper that creates a Sequence out of a BasicSequence. It's used as "aSequence=SequenceWrapper(aBasicSequence)" I'm not sure how necessary this is since "tuple(aBasicSequence)" would work as well or better in most situations. All names are, of course, open for debate. I think the general approach taken here and in Paul's file classes may be useful elsewhere: take a canonical builtin type, in this case tuple for immutable sequences. Define a wide protocol based on that, in this case Sequence. Find the narrowest interface that allows you to recreate that type, and define a protocol based on that, in this case BasicSequence. Finally define some helpers that help derive classes that obey the wide protocol and convert objects that obey the narrow protocol to the wide protocol, in this case SequenceBase and SequenceWrapper. I'll try the same with MutableSequences, starting with lists, and report back as to whether this approach works there as well. Later though. -tim ------=_NextPart_000_0361_01C0ACA1.59A4F970 Content-Type: text/plain; name="sequence.py" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="sequence.py" import types, operator class _iterable: "An iterable implements __getitem__ or the iteration protocol (TBD)" def __implementedby__(self, object): #if implements_iter_protocol(object): return 1 if hasattr(object, "__getitem__"): return 1 # This isn't really right, what we want to do is to check = whether # object[0] will work. However using the try: object[0] approach # man not work correctly. If the object uses the # current iteration protocol calling object[0] may # advance the iteration. if not type(object) =3D=3D types.InstanceType and \ operator.isSequenceType(object): return 1 return 0 Iterable =3D _iterable() =20 class _basicSequence: # This is an iterable, but it doesn't seem useful to inherit """A basic sequence implements __getitem__ and __len__ This is sufficient to build a tuple, but doesn't necessarily support concatenation, repitition or slicing. """ def __implementedby__(self, object): try: len(object) # Even though I refused to do this for the iterable = protocol, it's probably OK here. # Objects with a length attribute should not be changed by = accessing an item. object[0] except IndexError: # This indicates the indexing is OK, but the sequence length = is zero. pass except: return 0 return 1=20 BasicSequence =3D _basicSequence() class _sequence: # This is a sequence, but it doesn't seem useful to inherit """A full sequence implements the same methods as a tuple. A sequence can always be constructed from a BasicSequence, see SequenceWrapper below. =20 """ def __implementedby__(self, object): try: len(object) # I'm hoping this is less expensive for long sequences. 0*object + 0*object object[:0] # Even though I refused to do this for the iterable = protocol, it's probably OK here. object[0] except IndexError: # This indicates the indexing is OK, but the sequence length = is zero. pass except: return 0 return 1 Sequence =3D _sequence() # It's kind of obnoxious having both BasicSequence and SequenceBase. One = or both should # probably be renamed. class SequenceBase: """Class to simplify derivation of objects that satisfy the full = sequence protocol. This class provides the same functionality as a tuple. At a minimum, __getitem__ and __len__ must be overridden. = Concatenation and repition return tuples by default. If other behaviour is desired, override = __add__ and __mul__ as well. Subclasses may also want to override __getslice__ for efficiency. =20 """ # The following list is based on section 2.1.5 in the library = reference # x in s: Should be take care of by the sequence prototcol(?) # x not in s: Ditto # s + t: def __add__(self, other): return tuple(self) + tuple(other) # s * n, n * s: def __mul__(self, n): return tuple(self)*n __rmul__ =3D __mul__ # s[i] def __getitem__(self, i): raise NotImplementedError # s[i:j] def __getslice__(self, start, stop): l =3D [] for i in range(start, stop): l.append(self[i]) return tuple(l) # len(s) def __len__(self): raise NotImplementedError # min(s) # max(s) These two work automagically for any iterable. class SequenceWrapper(SequenceBase): def __init__(self, wrapped): self.wrapped =3D wrapped def __getitem__(self, i): return self.wrapped[i] def __len__(self): return len(self.wrapped) # Still to come, mutable sequences. # Very basic testing. class _iterable: def __getitem__(self, i): if 0 <=3D i < 5: return i raise IndexError anInterable =3D _iterable() class _basicSequence(_iterable): def __len__(self): return 5 aBasicSequence =3D _basicSequence() aSequence =3D SequenceWrapper(aBasicSequence) for obj, isIter, isBasic, isSeq in [((),1,1,1), ([],1,1,1), (anInterable, 1, 0, 0), (aBasicSequence, 1, 1, 0), (aSequence, 1, 1, 1)]: assert Iterable.__implementedby__(obj) =3D=3D isIter assert BasicSequence.__implementedby__(obj) =3D=3D isBasic assert Sequence.__implementedby__(obj) =3D=3D isSeq print obj, "(", isIter, isBasic, isSeq, ") passed" ------=_NextPart_000_0361_01C0ACA1.59A4F970-- From sverker.is@home.se Thu Mar 15 00:31:22 2001 From: sverker.is@home.se (Sverker Nilsson) Date: Thu, 15 Mar 2001 01:31:22 +0100 Subject: [Types-sig] Interface PEP References: <3AAFCBDF.39A0735D@home.se> Message-ID: <3AB00D5A.453E7BCF@home.se> Marcin 'Qrczak' Kowalczyk wrote: > > Wed, 14 Mar 2001 20:53:05 +0100, Sverker Nilsson pisze: > > > Well, what I can agree with is that the interfaces are _descriptions_: > > more for the human reader, than for (dynamic or static) type-checking. > > They are both for humans and for the interpreter / compiler. > > > I could consider the type of an object to be something different > > than the interface, but then _also_ different from the class, > > contrary to what you imply below. > > There are always problems with underspecified terminology. Especially > the area of types, classes, interfaces, subtyping / subclassing / > inheritance, polymorphism is really confusing... The same terms are > used in the context of different systems or languages with different > meaning. Easy to be confused or misunderstood. > > Python uses terms "type" and "class". There are several builtin > types. Object created as class instances all have the same type > (InstanceType), and can be further distinguished by their classes. > Many people agree that it would be cleaner to unify these concepts > in future. Though I'm sure you know more about this than me, I will still argue from my ignorance. Here we go: What not only many but perhaps all people would agree on, including me, is that the builtin types or classes should be unified with user-defined classes in this way: So that user-defined classes can inherit from the classes that the objects with builtin types have. That doesn't necessary mean unifying types with classes. It could just as well mean that we defined the classes of the builtin objects. I.E we would still have types.ListType that would not be an inheritable class, but we would also have e.g. BuiltinClasses.List. That I would think should solve the problem that people have with not being able to inherit. If it is cleaner or not, compared to making ListType a class, I wan't argue much about unless but saying that it is, well, arguable :-) Why would it be cleaner to unify type and class, given the many variants that exist, that you describe (some of) yourself below? > > C++ uses terms "type" and "class". There is a similarity to Python's > usage, but in C++ there is no single InstanceType: some types are > classes, some are not classes. Class templates aren't types yet. Yes C++ I know some about; that didn't stop me from, for some reason, considering types as naturally distinct from classes... maybe because I've been using Python so much lately! > > Eiffel uses terms "type" and "class". Class is the basic unit of > modularity (how an object is defined), type describes an interface > (what do we expect from an object). Each non-parametrized class > yields a type. Each parametrized class may yield a type when applied > to appropriate arguments. Humm... thanks for explanation... below you say that Eiffell had problems with having to analyze too much to see if a class would yield a subtype. But here you seem to say that type is separate from class, and it describes an interface. So the problems you say they had, seem to be the same whether they called the type a type or interface. The problem might have to do with that the class 'yields a type'. It would be yielding an interface, with other terminology. I think you said this was the wrong approach in the other mail. But maybe this is what dynamically typed languages want to do. > > Haskell and Clean use terms "type" and "class". Type defines an > implementation and functions often require a concrete type. Class > defines an interface. Types may conform to classes. Functions may also > require that the type of their argument conforms to some classes. This > is quite backwards compared to Eiffel and OO theoretic terminology. > Parametrized types are unified with types, i.e. types of concrete > objects are special cases of parametrized types in general. I used Haskell some time but gave up partly because I felt the type system was constraining and I was afraid of running into walls long into the project if things wouldn't fit together as planned... as they never do. Of course statically typed shares some problems; easy refactoring was mentioned, perhaps on this list, as a big advantage with dynamically typed languages. Haskell is arguably one of the more sofisticated. Still they had problems defining general container classes, at least before they introduced dual inheritance. (I don't know if that's there now officially, I remember it was controversal.) And I read some paper that showed how impossible it was to define a natural join... in some context. > > OCaml uses terms "type" and "class". Type defines an implementation and > functions often require a concrete type. The OO subsystem introduces > classes (templates of objects with method definitions) and class types > (interfaces of objects with method signatures). A class type is a type. > A class is something between an object and a type. I hope I got it > right - I don't have an experience with OCaml's OO subsystem. I'd be happy to use OCaml for its superb speed, have looked just a little in it, but may be held back afraid of future refactoring problems due to the static type system. BTW, does it have any dynamic types at all? > > We have plenty of definitions to choose from :-) Right :-) > > > > Interfaces". One class, or "type" of object, can implement multiple > > > > So you identify the object's class with its type... why? > > In Python they have the same purpose. A type, combned with a class > (in case of InstanceType), describe the implementation of an object. > Well, partially: object attributes can be different for each object. Ok well, I thought they as well described the interfaces, at least partially. > > Interfaces of objects are not described formally in Python nor checked. Not formally but informally. I think it is quite common to check if an object has a given method, and not care about its class. You can never know all about an interface, can you? I'd say there is always a matter of degree how much you know. Types or interfaces are open-ended: from specifying Integer, in range 1..10, to is_prime(x), to... whatever. Since it's open-ended, I would say that we already know something about interfaces. We know what interface the built-in objects use. We know the basics about any class's interface: they all either respond to a method by executing some code or raise the AttributeError. Since we know something, it's about being able to know more, and to be able to extend it in the future, preferrably on user level. That leads to me arguing that InstanceType is the type of a class that you don't know much about, except that it's a class. It's just a special case. Extending our knowledge could be made by having the class type be something more specific than InstanceType. Possibly the type could be dynamically generated and contain a pointer to the object, to allow for generality in checking that the methods are compatible with some other type (interface). But it could also just be a declaration saying that 'I am a sequence' and then we should be able to trust that it has those methods... we can't prove it anyway. Just checking that it's got the methods doesn't help because they may all just be null methods. I think it should be useful to be able to declare types/interfaces that claim... well anything, like 'I am a prime'. > One of current goals is to change this. Right... actually I came into this discussion more or less by accident so well, it's their or your's goal :-) > > > Well I'd say "lots of different _classes_ of objects.." > > Not types of - that would again prematurely fix what we mean > > with the type of an object. > > I assume that we use the term "type" as currently defined in Python, > and don't change that. Now it just means the types defined in types.py. Right? Are you saying that no more types should be added, except by new builtin objects? And are you saying that we cant add more functionality for the types, like type.is_supertype(other)? I mean, is that because that would change the meaning of the term "type"? Or is it because of that the builtin type(object) could return something else than InstanceType? - if the designer of the class so whanted. Would that redefine the meaning of "type" from how it is currently defined, and if so, in some bad way? Or what do you mean you are assuming? (Sorry if the questions above look rethoric, they are not :-) > > > I'd say SequenceType or MappingType should be types just as well > > as ListType. Why make them different kinds of things? > > ListType describes an implementation, i.e. how an object was > constructed. There is a particular object layout and particular > methods. When something has type ListType you know its particular methods. So why doesnt ListType define an interface too? It also defines a particular object layout as you say. I'd say that should be considered a part of the interface, too. Only a part that most Python functions wouldn't care about - relevant only for extension functions and the compiler. But a relevant interface specifcation, nevertheless. So I don't see any significant difference, just the additional interface specification for the implementation/object layout. > > SequenceType describes an interface, i.e. how an object can be used. How an object can be used, isn't that exactly also what ListType describes? I mean, not explicitly, but knowing it is a ListType, you know implicitly all about how it can be used. Making it explicit, e.g. by defining a function type->description, shouldn't change what a type is. > An interace can include things that "len(o) is defined and means blah > blah" - it doesn't matter how len is dispatched. Sure. But why couldn't it include a specification of how it's dispatched too, if it wanted for some reason. > > > I'd say ListType is a _subtype_ of SequenceType. > > A basic misunderstanding of some languages is confusing subtyping > (the ability to use one type if another is required, i.e. a relation > on interfaces) and inheritance (basing a class definition on another > class, i.e. reusing the implementation). So if I am understanding you correctly: The problem is you get a type (= class) that is based on a particular implementation? And not being able to control the interface separately from the implementation? Yes in C++ there is no interface concept separate from classes, right? And the type is the same as the class. But I was separating the type from the class. > > Eiffel has problems because of this. It requires checks of the whole > assembled system to see if a subclass yields a subtype or not. See previous comment about your description of Eiffel. Isn't the problem really that it tries to yield an interface (or type, whatever we call it)? I can believe there could be problems doing that statically and optimally. I'm not arguing much about that now because it wasn't my idea originally to have an __interface__ field in the objects. Although I think that it could be good to have. I just wanted to call it the type instead. This problem with Eiffell seems irrelevant to that. > > C++ and Java have lack of expressiveness because of this. They > require some casts because a method signature can't change in an > inherited class. > > I don't say that you claim that subtyping is the same as inheritance. Right. > But you try to unify types / classes with interfaces, which doesn't > work well. No I wasnt trying to unify to 1 concept. Just to 2 concepts: types and classes. I wasn't aware that types must/should be the same as classes. We have types and classes. Why join them and then introduce a new concept, interface? I'd like to improve functionality of types, by unifing them with interfaces but keep them separate from classes. Do we need 3 concepts - types, interfaces, and classes? > > A type / class may generate an interface requirement: namely that an > object is of this class or a subclass (the "or a subclass" part is > very problematic). An interface may generate a type / class definition: > we may want to hold an arbitrary object conforming to an interface. > They are sometimes blurred, but they are definitely different concepts. I think this problems disappears since I am not talking about unifying classes with interfaces, just types with interfaces, and divorcing types and classes. > > We can have two interfaces (e.g. "comparable" and "convertible to > a string"). One object conforms to the first but not to the second, > another conforms to the second but not to the first. This situation > cannot be described by locating the object's type in the interface > tree and checking its supertypes. It's not a tree. Same here, I think, no problem because type != class, although I am not sure I am following you in this example. Sverker From sverker.is@home.se Thu Mar 15 02:41:33 2001 From: sverker.is@home.se (Sverker Nilsson) Date: Thu, 15 Mar 2001 03:41:33 +0100 Subject: [Types-sig] Interface PEP References: <3AAF7016.7164FDC2@home.se> Message-ID: <3AB02BDC.40E3E2B@home.se> Marcin 'Qrczak' Kowalczyk wrote: > > Wed, 14 Mar 2001 14:20:22 +0100, Sverker Nilsson pisze: > > > The type() builtin would return, as usual, InstanceType if the > > __type__ special attribute was not defined. Otherwise it would return > > what __type__ returned. - which would be a user defined type (aka > > interface) or even a builtin type, if the class wants to claim it > > emulates a built-in type. > > Warning, warning! This is a wrong way. > > There is no concept of asking an object which interface it implements. > There is no "the" interface it implements. That's something that seems to be suggested in the PEP that this thread concerns. http://www.zope.org/Members/michel/InterfacesPEP/PEP.txt Quoting from that: """ Interface Assertion The next step is to put classes and interfaces together by creating a concrete Python class the asserts that it implements an interface. Here is an example "FishMarket" component that might do this: [...] class FishMarket implements FishMarketInterface: [...] The interface assertion provided in a 'class' statement like this is stored in the class's '__implements__' class attribute. After interpreting the above example, you would have a class statement that can be examined like this: >>> FishMarket >>> FishMarket.__implements__ (,) """ So for the record, it was not my idea with having explicitly specified interfaces, though I may agree with it. I was just considering calling the 'interface' 'type'. If it was any misunderstanding of that, I'll leave it to Michel Pelletier or some other to respond to the rest of your mail, for now. Regards, Sverker Nilsson From pedroni@inf.ethz.ch Thu Mar 15 03:53:32 2001 From: pedroni@inf.ethz.ch (Samuele Pedroni) Date: Thu, 15 Mar 2001 04:53:32 +0100 Subject: [Types-sig] Interface PEP References: <3AAF7016.7164FDC2@home.se> <3AB02BDC.40E3E2B@home.se> Message-ID: <009f01c0ad03$8239b120$395821c0@newmexico> Hi. It seems there is a bit of confusion. I think the following issues are quite orthogonal: 1) the purely runtime mechanism for enforcing args and return value _domain_ checking under devel. 2) the kind of "type" systems that can be used or experimented with 1) 3) intefaces as defined by the PEP [mostly a summary, at least I hope] 1) with syntactic sugar (that for the moment we do not have): def f(a: D1,b: D2) -> D3 ... return r will test at runtime whether a,b on entry and r on return pass: D1.__implementedBy__(a) D2.__implementedBy__(b) and D3.__implementedBy__(c) so D1,D2,D3 are "algorithmic" hints/definitions of the domain, codomain of f 2) it would be nice to have a set of predefined domain checks and domain constructors ideally they would build up a type system, but 1) in principle enable any kind of decision and can even be used mixing up different paradigms. A possible goal and test-case is to annotate in such way the std lib. Now the algorithm in the standard lib work has long as the inpunt support the needed operations (protocols), at the moment python does not require that inputs inherit from abstract classes or explicitly declare to implement some interfaces: e.g. simply as long as an object supports __getitem__ it is a valid input to: def first(a): return a[0]. So the domain for a is "supporting []" (being a SequenceType or implementig __getitem__). To be appropriate for annotating the std lib. domains must algorithmically (and 1) allows for that ;)) check if a protocol is implemented. To enforce that inputs explicitly declare to implement an interface *does not* correspond to the actual practice and potentially would require a lot of changes in the existing code around that uses the interface. OTOH the latter kind of check wheter an input explicitly implements an interface can be implemented and fits with 1, and like Tim Hochberg has proposed having wrapper or abstract classes to enforce a protocol is also ok and nice for future code. 3) The idea of interface as proposed enforces and documents design and somehow enforces what we have called a protocol (implementing a set of ops (methods)) and is orthogonal with 1): * one can check wheter an input explicitly declare to implement an interface (interfaces could have their __implementedBy__) * ideally there could be a constructor that produce a protocol domain checker out of an interface: JustProtocol(FooInterface).__implementedBy__(baz): checks whether baz declares to implement FooInterface (this for speed), or checks wheter baz defines the methods required by FooInterface * if we endup having signatures for functions (def f(a: D1) -> D2) interfaces could be defined with them and check for them too. Note: protocols as intended do not form an explicit hierarchy, but there is an inclusion relation. regards Samuele Pedroni From paulp@ActiveState.com Thu Mar 15 06:36:00 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Wed, 14 Mar 2001 22:36:00 -0800 Subject: [Types-sig] Interface PEP References: <3AAF7016.7164FDC2@home.se> <3AB02BDC.40E3E2B@home.se> <009f01c0ad03$8239b120$395821c0@newmexico> Message-ID: <3AB062D0.19EDD38F@ActiveState.com> Samuele, I agree with the vast majority of your analysis but I'm not sure that the objects you call Protocol Domain Checkers and Interfaces are necessarily distinct. An interface has two parts: there is the "semantic" part that is checked only by human inspection and there is the signature part that could, in theory, by checked by software at runtime (or even compile time in some cases). A "domain checker" inevitably also has a semantic part and an operational part. The operational part also checks an object for conformance to a set of rules. I don't see how the two things are very different other than the syntax. > * one can check wheter an input explicitly declare to implement an > interface (interfaces could have their __implementedBy__) I've come to think that there is a difference between what Michel calls __implementedby__ and what I now call __check__ (as Tim H. called it originally). Both of our models should support both methods. One checks that an object CLAIMS to support an interface and the other checks that it has the right signature. > * ideally there could be a constructor that produce a protocol domain > checker out of an interface: > JustProtocol(FooInterface).__implementedBy__(baz): > checks whether baz declares to implement FooInterface (this for speed), > or checks wheter baz defines the methods required by FooInterface I would write that as FooInterface.__check__(baz) >... > Note: protocols as intended do not form an explicit hierarchy, but there is an > inclusion > relation. If you integrate protocols and interfaces then it makes sense for the integrated thingee to have an hierarchy. -- Take a recipe. Leave a recipe. Python Cookbook! http://www.activestate.com/pythoncookbook From paulp@ActiveState.com Thu Mar 15 07:33:55 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Wed, 14 Mar 2001 23:33:55 -0800 Subject: [Types-sig] Another cut at Sequence declarations References: <3AAE220A.DC92ABED@ActiveState.com> <036401c0acdc$0617a790$87740918@CX781526B> Message-ID: <3AB07063.22312F55@ActiveState.com> I think you're making good progress Tim! I'd appreciate it if you could finish the major protocols: Sequence (Mut/Unmu), Mapping (Mut/Unmu), ReadFile, WriteFile. I'll move the interfaces you define into my module and we'll put the base classes into some kind of basic types module for clarity. -- Take a recipe. Leave a recipe. Python Cookbook! http://www.activestate.com/pythoncookbook From paulp@ActiveState.com Thu Mar 15 07:41:07 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Wed, 14 Mar 2001 23:41:07 -0800 Subject: [Types-sig] Type equivalence References: <3AAED59A.AB2AB4AA@ActiveState.com> <15023.44909.286985.224674@anthem.wooz.org> Message-ID: <3AB07213.E025201D@ActiveState.com> Marcin 'Qrczak' Kowalczyk wrote: > > ... > > We may need to add a boolean type to Python. > > I agree. But I'm afraid that if Guido wanted it, he would add it > earlier. It would be easy enough to add it but it depends on whether we want the definition of lambda obj: obj in (0, 1) or the definition that "every object can be used as a boolean": lambda obj: "you betcha!" > Indeed. The system must not require ReadableFile to know about > MustHaveRead. Let's not repeat Java's mistakes. The system as we have been designing it does not have a problem with this sort of thing. It works like this: 1. First it checks if an object *declares* an interface. If it can't find that it falls back to: 2. Checking the signature of the objects. In some future "strict types" mode, this sort of "passing off" might issue a warning. I claim, though, that it won't arise that much in practice. It is going to be very rarely the case that two different modules define methods with the SAME name and the SAME semantic without some reference to a common defining document or interface. That defining thing is the type that the other two are subtypes of. In other words, the only reason you can safe pass around things with .read() to modules from company A and B without a problem is because Guido defined what read() means a long time ago informally. I'd be interested to hear about an example where the same method name arose with the same signature and semantics completely independently. -- Take a recipe. Leave a recipe. Python Cookbook! http://www.activestate.com/pythoncookbook From paulp@ActiveState.com Thu Mar 15 07:44:21 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Wed, 14 Mar 2001 23:44:21 -0800 Subject: [Types-sig] File Protocol Proposal References: <3AAED59A.AB2AB4AA@ActiveState.com> <15023.44909.286985.224674@anthem.wooz.org> <025d01c0acb4$faef6e60$87740918@CX781526B> Message-ID: <3AB072D5.A258C249@ActiveState.com> Tim Hochberg wrote: > > ... > > It's possible that this has changed, but according so section 2.1.1 of the > Library Reference, everything is a boolean type! Or am I missing something. That's true. It's also true that boolean functions in Python basically always return 1 or 0 so there are really two different conventions at work... > The simple way is to just use union type. I called this composite in my > orginal proposal, but that may be bad terminology. Anyway, the following > should work: > > def weeble(wobble : UnionOf(NoneType, String)): > # .... Agreed. This is the right way. > If we forced typecheck objects to all inherit from a common base class, it > would also be pretty easy to write this as: > > def weeble(wobble : NoneType | String): > # .... Agreed on this too. But I was going to use the "or" keyword if at all possible. I admit that the pipe symbol is more mathematically conventional but "or" is more, er, English. -- Take a recipe. Leave a recipe. Python Cookbook! http://www.activestate.com/pythoncookbook From paulp@ActiveState.com Thu Mar 15 07:59:35 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Wed, 14 Mar 2001 23:59:35 -0800 Subject: [Types-sig] Interfaces API Message-ID: <3AB07667.BB8C3F9A@ActiveState.com> We'll need to agree on an API for interfaces. My API is very simple: interface Interface: """The shared behavior of interface objects""" def __check__(self, obj) Michel's is a little more "feature rich": interface InterfaceInterface(InterfaceBaseInterface): """ This is the interface to interface objects that are described in this document. """ def getBases(): """ Returns a sequence of base interfaces this interface extends. """ def extends(other): # convenience function? """ Does this interface extend the 'other' interface? """ def isImplementedBy(object): # convenience function? """ Does the given object implement the interface? """ # convenience function? def isImplementedByInstancesOf(klass): """ Do instances of the given class implement this interface? """ # do you differentiate between required # and optional names? I ask because it seems like it # would make sense to have docstrings and other # metadata for optional methods etc. def names(): """ Return the attribute names defined by the interface """ def namesAndDescriptions(): """ Return the attribute names and description objects defined by the interface """ # convenience function? def getDescriptionFor(name, default=None): """ Return the attribute description object for the given name """ # I think deferred classes were taken out of the PEP def deferred(): """ Return a deferred class corresponding to the interface. """ I'm surprised that you don't have the equivalent of my __check__. Given an object don't we want a way to ensure that it really does have the right names, methods with the right parameters and so forth? In my mind, __check__ is the major feature required for runtime checks. The other stuff is of lesser interest ot me. Also, I wonder if an API could get into the core language with CamelCaseMethodNames. Guido Isn't A CamelCaseGuyAfterAll. I really prefer camel case myself but what's the point in fighting the BDFL on religious issues? (I'll point out, however that camel case names would have a consistent syntax for hasAttr and hasKey, which Python does NOT!) -- Take a recipe. Leave a recipe. Python Cookbook! http://www.activestate.com/pythoncookbook From barry@digicool.com Thu Mar 15 14:20:03 2001 From: barry@digicool.com (Barry A. Warsaw) Date: Thu, 15 Mar 2001 09:20:03 -0500 Subject: [Types-sig] Re: Type equivalence References: <3AAED59A.AB2AB4AA@ActiveState.com> <15023.44909.286985.224674@anthem.wooz.org> <3AB07213.E025201D@ActiveState.com> Message-ID: <15024.53139.238620.215002@anthem.wooz.org> >>>>> "PP" == Paul Prescod writes: PP> It would be easy enough to add it but it depends on whether we PP> want the definition of PP> lambda obj: obj in (0, 1) PP> or the definition that "every object can be used as a PP> boolean": PP> lambda obj: "you betcha!" The type/class that I've prototyped (in either a C or Python module) defined two objects `true' and `false' which mostly supported comparison semantics with current true and false objects in Python. E.g. true == 1, but true is not 1, and false == {} but false is not {}. I say mostly because it's not completely possible to do the right thing qpre-richcomps. I could do some digging and update the code. PP> In some future "strict types" mode, this sort of "passing off" PP> might issue a warning. I claim, though, that it won't arise PP> that much in practice. Maybe, but I really don't know. Python's typing system will undoubtably have its own unique quirks, and "normal usage patterns" cannot be fully predicted. That's why we need to get some experience with a working implementation. -Barry From Samuele Pedroni Thu Mar 15 14:49:36 2001 From: Samuele Pedroni (Samuele Pedroni) Date: Thu, 15 Mar 2001 15:49:36 +0100 (MET) Subject: [Types-sig] Interface PEP Message-ID: <200103151449.PAA00252@core.inf.ethz.ch> Hi. > Samuele, I agree with the vast majority of your analysis > but I'm not > sure that the objects you call Protocol Domain Checkers and Interfaces > are necessarily distinct. Maybe is just matter of viewpoint. As fas as I have understood with Michel interfaces the idea is that one declare explicitly when a class implements them and you can check for that looking at the __implements__ attr, which is setup at class definition time after enforcing the "implements" decl through a check. All interfaces are domain but at least at the beginning of our discussion not all domains were interfaces, one could define the domain Prime. So I don't know if the interface syntax can be stretched to the point to deal with that too and Prime is more an algorithmic-checkable property than something that an input should declare to implement. OTOH yes as far as protocol are concerned one could use the interface syntax to define them. So we can the following: interface Readable: def read(size) than there is an ambiguity problem when we should enforce the check: with your new distinction __check__ vs. __implementedby__ should: def grab(r: Readable) ... test Readable.__check__(r) or Readable.__implementedby__(r). I think we have agreed that for backw-compatibility and for python tradition so far the right kind to check is the first (e.g. at least for the std lib). But a syntactic way to ask for checking the second would also be nice, because one may require that an input really explicitly declare an interface because this is the only way to be sure that not also the signatures match but all the implicit requirements that are associated with the interface are met. We have the following distinction and inclusions: interfaces < protocols < domains So maybe we need a protocol syntax to define protocols, or a syntax to distinguish the two kind of checks. Domain can be defined just as singletons with the right method. > > If you integrate protocols and interfaces then it makes sense for the > integrated thingee to have an hierarchy. > So the point is that I don't know to which extent is a clean thing to integrate the two. regards, Samuele Pedroni. From qrczak@knm.org.pl Thu Mar 15 15:15:35 2001 From: qrczak@knm.org.pl (Marcin 'Qrczak' Kowalczyk) Date: Thu, 15 Mar 2001 16:15:35 +0100 (CET) Subject: [Types-sig] Re: Type equivalence In-Reply-To: <15024.53139.238620.215002@anthem.wooz.org> Message-ID: On Thu, 15 Mar 2001, Barry A. Warsaw wrote: > The type/class that I've prototyped (in either a C or Python module) > defined two objects `true' and `false' which mostly supported > comparison semantics with current true and false objects in Python. > E.g. true == 1, but true is not 1, and false == {} but false is not {}. This is strange. Why not just have true and false as canonical truth values, without magic in comparisons, and keeping old semantics of which values are considered true? The biggest problem is that letting == < etc. return false/true instead of 0/1 breaks compatibility a lot. -- Marcin 'Qrczak' Kowalczyk From paulp@ActiveState.com Thu Mar 15 16:14:45 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Thu, 15 Mar 2001 08:14:45 -0800 Subject: [Types-sig] Type Annotations Progress report Message-ID: <3AB0EA75.554BF701@ActiveState.com> Before I documented everything I wanted to do a real-world test. So I added type annotations to the os module. I even added a new type that was local to that module for handling P_WAIT, P_NOWAIT. I'd appreciate if people would at least take a cursory peek at the os module, if not the implementation. OS: www.prescod.net/pytypes/os.py Implementation: http://www.prescod.net/pytypes/typecheck.py I'm going to release it with one big known limitation. The stack traces it produces are very useful if you know how to read them but somewhat ugly compared to what they will be eventually. I'm trying to figure out how to fix that... Remember that this is experimental, transitional, prototype, type annotation system. The syntax is not ideal. Nevertheless, I think it is quite simple, easy to use and immediately useful. Try making a small program that calls some of the "os" functions. Tell me if any of them fail when they should not. For instance, this is what Python *used* to do when you made a mistake in os.environ: C:\data\research\pytypes\peps\src>python bad.py Traceback (most recent call last): File "bad.py", line 2, in ? os.getenv(5) File "c:\python20\lib\os.py", line 321, in getenv return environ.get(key, default) File "c:\python20\lib\os.py", line 298, in get return self.data.get(key.upper(), failobj) AttributeError: 'int' object has no attribute 'upper' This is what it does NOW: C:\data\research\pytypes\peps\src>python bad.py Traceback (most recent call last): File "bad.py", line 2, in ? os.getenv(5) File ".\os.py", line 437, in getenv __paramcheck__() File ".\typecheck.py", line 167, in __paramcheck__ raise InterfaceError(name, obj, type) typecheck.InterfaceError: Parameter 'key', expected Unicode or 8-bit string. Instead it got '5' () That's already an improvement! Instead of getting an error about a method I've never heard of on an object I may never of heard of on a random line of os.py, I got an obvious parameter error with a name from the parameter list and the erroneous value I passed it. This is what it should do when I have cleaned everything up: Traceback (most recent call last): File "bad.py", line 2, in ? os.getenv(5) File ".\os.py", line 433, in getenv def getenv(key, default=None): InterfaceError: Parameter 'key', expected Unicode or 8-bit string, got '5' Instead it got '5' () If anybody knows how I can control the traceback associated with the exception better (especially to eliminate one or more levels of traceback), I'd appreciate it. One trick is to rewrite in C. I'd like something less drastic while I'm still experimenting. :) Thanks for all of the advice I got from people! -- Take a recipe. Leave a recipe. Python Cookbook! http://www.activestate.com/pythoncookbook From tim.hochberg@ieee.org Thu Mar 15 17:01:18 2001 From: tim.hochberg@ieee.org (Tim Hochberg) Date: Thu, 15 Mar 2001 10:01:18 -0700 Subject: [Types-sig] Another cut at Sequence declarations References: <3AAE220A.DC92ABED@ActiveState.com> <036401c0acdc$0617a790$87740918@CX781526B> <3AB07063.22312F55@ActiveState.com> Message-ID: <053001c0ad71$8d733020$87740918@CX781526B> From: "Paul Prescod" > I think you're making good progress Tim! I'd appreciate it if you could > finish the major protocols: Sequence (Mut/Unmu), Mapping (Mut/Unmu), > ReadFile, WriteFile. I'll move the interfaces you define into my module > and we'll put the base classes into some kind of basic types module for > clarity. I'll continue plugging away. Unsuprisingly Mutable sequences are turning out to be more of pain than I had naively hoped. First off, the minimal interface to simulate a list appears to be: __getitem__, __len__, __setitem__, __delitem__ and either insert or __setslice__ Or, I just realized, a __setitem__ that dealt correctly with slice objects would also work. So, eitther insert, __setslice, or __setitem__ (sometimes) work. Ick! This makes it apparent that there are really two MutableSequences. A MutableSequence and an ResizableMutableSequence. A Numeric array is an example of the former, a list of the later. I'm not entirely convinced that defining a mutable sequence type or types will turn out to be worth the trouble, I think that most sequences passed to functions are used immutably. I'll keep working on this though and see what I come up with. Another issue. Anyone have any ideas on how to nondestructively test for __setitem__? I have a few clunky ideas, but none that I like. This brings me to my final observation: this would be immensely easier with type-class unification. If every object implemented __setitem__, etc instead of the tp_setitem it would help in two ways. First it would be trivial to query whether a given object supported, e.g,m __setitem__. Second, it would make it easy to define anonymous protocols. This would take a lot of pressure of us to design a complete type system, we could focus on the common cases. The above MutableSequence might be defined anonymously as: def wibble(aMSeq : ('__getitem__', '__len__', '__setitem__')): or maybe even: def wibble(aMSeq: BasicSequence & ('__setitem__')) OK, one more observation: it may be worth our while to encapulate whatever we come up with for faking these checks as functions. In fact, I guess we could just implement them as protocol checkers and then write the above as either: import typecheck as tp def wibble(aMSeq : tp.BasicSequence & tp.__setitem__) def wibble(aMSeq : tp.UnionOf(tp.BasicSequence, tp.__setitem__) Now I'm starting to ramble, so I'll sign off. -tim From Samuele Pedroni Thu Mar 15 17:06:20 2001 From: Samuele Pedroni (Samuele Pedroni) Date: Thu, 15 Mar 2001 18:06:20 +0100 (MET) Subject: [Types-sig] Interfaces API Message-ID: <200103151706.SAA04488@core.inf.ethz.ch> > > We'll need to agree on an API for interfaces. The proposals express two orthogonal point of view that cannot simply be merged, one can be intended as an extension/restriction of the other, but IMHO they cannot be flatten down to the same thing. > My API is very simple: > > interface Interface: > """The shared behavior of interface objects""" > def __check__(self, obj) This is the minimal requirement for something that I would call a domain. > > Michel's is a little more "feature rich": > > I'm surprised that you don't have the equivalent of my __check__. Given > an object don't we want a way to ensure that it really does have the > right names, methods with the right parameters and so forth? In my mind, > __check__ is the major feature required for runtime checks. > Python Cookbook! http://www.activestate.com/pythoncookbook because as far as I have understood: the semantic of the following definition is that: class A implements I: ... should enforce through a check that A offers all the signatures required by I. And conventionally expresses that A meet all of the implicit requirements (not at signature level) that come with I. regards, Samuele. From tim.hochberg@ieee.org Thu Mar 15 19:00:49 2001 From: tim.hochberg@ieee.org (Tim Hochberg) Date: Thu, 15 Mar 2001 12:00:49 -0700 Subject: [Types-sig] Type Annotations Progress report References: <3AB0EA75.554BF701@ActiveState.com> Message-ID: <057f01c0ad82$4017f700$87740918@CX781526B> Paul Prescod writes: > Before I documented everything I wanted to do a real-world test. So I > added type annotations to the os module. I even added a new type that > was local to that module for handling P_WAIT, P_NOWAIT. I'd appreciate > if people would at least take a cursory peek at the os module, if not > the implementation. I've taken only a quick peak at the implementation, and here are a few small comments based on that peak. First, since there is a common Interface class, should we add intersection and union as & and |. I know you prefer 'and' and 'or', but the former we can do now, the later has to wait for language support. Also, I see that CheckInterface, if passed a tuple or a list as in interface, checks each item in the checked object against each item in the interface. For those playing along at home who found that statement completely meaningless: val : (IString, IInteger, IFloat) Checks that val[0] is an IString, val[1] is an IInteger and val[2] is an IFloat I like this. So, should using a tuple imply something different about val than using a list (mutable sequence vs immutable sequence). I'm leaning toward not, but I figured I'd bring it up. On the third hand, the above syntax puts a kink in my hope of using & and | since: val : (IString, IInteger) | (IFloat, IFloat) won't work. Harumph. -tim From paulp@ActiveState.com Thu Mar 15 19:51:20 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Thu, 15 Mar 2001 11:51:20 -0800 Subject: [Types-sig] Type Annotations Progress report References: <3AB0EA75.554BF701@ActiveState.com> <057f01c0ad82$4017f700$87740918@CX781526B> Message-ID: <3AB11D38.551EC6BF@ActiveState.com> Tim Hochberg wrote: > >... > On the third hand, the above syntax puts a kink in my hope of using & and | > since: > > val : (IString, IInteger) | (IFloat, IFloat) > > won't work. Harumph. You're right, it seems we can only use first-class Python syntax for tuple-building OR unions but not for both. :( -- Take a recipe. Leave a recipe. Python Cookbook! http://www.activestate.com/pythoncookbook From michel@digicool.com Thu Mar 15 21:34:20 2001 From: michel@digicool.com (Michel Pelletier) Date: Thu, 15 Mar 2001 13:34:20 -0800 (PST) Subject: [Types-sig] Interface PEP In-Reply-To: <3AB062D0.19EDD38F@ActiveState.com> Message-ID: On Wed, 14 Mar 2001, Paul Prescod wrote: > I've come to think that there is a difference between what Michel calls > __implementedby__ and what I now call __check__ Just an aside, I don't call anything __implementedby__, I propose __implements__. I think the former is semanticly ambiguous. -Michel From michel@digicool.com Thu Mar 15 21:45:20 2001 From: michel@digicool.com (Michel Pelletier) Date: Thu, 15 Mar 2001 13:45:20 -0800 (PST) Subject: [Types-sig] Interfaces API In-Reply-To: <3AB07667.BB8C3F9A@ActiveState.com> Message-ID: On Wed, 14 Mar 2001, Paul Prescod wrote: > We'll need to agree on an API for interfaces. My API is very simple: > > interface Interface: > """The shared behavior of interface objects""" > def __check__(self, obj) Well, i'm not sure how yours works. Is there a description somewhere? I'm wondering if you're interface objects describe their elements are attributes of that object. For example, in your model: interface Foo: def bar(): pass Does this make a 'Foo.bar'? We don't go that way, because we don't think there's any usefulness in it. The interface of an interface is *not* the interface that it describes, in our model. > Michel's is a little more "feature rich": > I'm surprised that you don't have the equivalent of my __check__. I don't know what it does. > Given > an object don't we want a way to ensure that it really does have the > right names, methods with the right parameters and so forth? Oh, I see. yes we have a method for this, but it is not part of the InterfaceInterface because we don't think it's that important. > In my mind, > __check__ is the major feature required for runtime checks. The other > stuff is of lesser interest ot me. But checking for the right number of names, methods, parameters, and even their types, is only a minor Java-esque win. There's certainly no guarantee that the implementation will work in any way at all the way its interface describes, and no way to test that quickly. For that, unit tests are ideal. Actually, we've been thinking about that alot and we've think that interfaces + unit tests are pretty damn cool. Anyway, about your second point, we have the complete opposite philosophy: run time checking is of lesser interest to us. Flexible, rich introspection for tools and programmers to address the stated goals in the PEP is our main focus. Not run time checking. > Also, I wonder if an API could get into the core language with > CamelCaseMethodNames. Guido Isn't A CamelCaseGuyAfterAll. I really > prefer camel case myself but what's the point in fighting the BDFL on > religious issues? (I'll point out, however that camel case names would > have a consistent syntax for hasAttr and hasKey, which Python does NOT!) I'm not emotionally commited either way. ;) Any case Guido recommends is fine with me. -Michel From paulp@ActiveState.com Thu Mar 15 21:48:09 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Thu, 15 Mar 2001 13:48:09 -0800 Subject: [Types-sig] Interface PEP References: Message-ID: <3AB13899.9AFDB33@ActiveState.com> Michel Pelletier wrote: > > On Wed, 14 Mar 2001, Paul Prescod wrote: > > > I've come to think that there is a difference between what Michel calls > > __implementedby__ and what I now call __check__ > > Just an aside, I don't call anything __implementedby__, I propose > __implements__. I think the former is semanticly ambiguous. You have isImplementedBy. But built-in Python features do not use camelcase and methods that are magically invoked are typically __ prefixed. -- Take a recipe. Leave a recipe. Python Cookbook! http://www.activestate.com/pythoncookbook From michel@digicool.com Thu Mar 15 21:53:38 2001 From: michel@digicool.com (Michel Pelletier) Date: Thu, 15 Mar 2001 13:53:38 -0800 (PST) Subject: [Types-sig] Interface PEP In-Reply-To: <3AB13899.9AFDB33@ActiveState.com> Message-ID: On Thu, 15 Mar 2001, Paul Prescod wrote: > Michel Pelletier wrote: > > > > On Wed, 14 Mar 2001, Paul Prescod wrote: > > > > > I've come to think that there is a difference between what Michel calls > > > __implementedby__ and what I now call __check__ > > > > Just an aside, I don't call anything __implementedby__, I propose > > __implements__. I think the former is semanticly ambiguous. > > You have isImplementedBy. Yes, it's not ambiguous in that case . Keep in mind, isImplementedBy does not try to "check" anything about the object you pass it other than it defines an __implements__ and the interface finds itself in that assertion. There is no rigorous checking to make sure the object isn't lying. This is a feature. > But built-in Python features do not use > camelcase and methods that are magically invoked are typically __ > prefixed. You lost me, isImplementedBy is not magically invoked. You have to specificly ask: FooInterface.isImplementedBy(fooObject) __implements__ on the other hand, is supposed to be the interface analog to __bases__. -Michel From paulp@ActiveState.com Thu Mar 15 22:06:21 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Thu, 15 Mar 2001 14:06:21 -0800 Subject: [Types-sig] Interfaces API References: Message-ID: <3AB13CDD.96D1D4DA@ActiveState.com> Michel Pelletier wrote: > >... > > Well, i'm not sure how yours works. Is there a description somewhere? There's the code. :) A more formal description will come later. http://www.prescod.net/pytypes/os.py http://www.prescod.net/pytypes/typecheck.py > I'm wondering if you're interface objects describe their elements are > attributes of that object. No, not yet. I see that as a feature I would add in a merger with your interface objects. > For example, in your model: > > interface Foo: > def bar(): > pass > > Does this make a 'Foo.bar'? I have no such syntax. I am happy to defer to your syntax. All I have are __check__ methods. If your objects can export some ability to verify that an instance could reasonably conform (in signature at least) to an interface, you can call it __check__ and we will be in sync. Or I could call mine implementedBy but I don't think that will fly. >... > But checking for the right number of names, methods, parameters, and even > their types, is only a minor Java-esque win. There's certainly no > guarantee that the implementation will work in any way at all the way its > interface describes, and no way to test that quickly. One of the constantly repeated requirements of the Python community (e.g. Barry Warsaw's post from a few days ago) is that if an object *looks like a file* it should be allowed to pass as a file through type checks. http://mail.python.org/pipermail/types-sig/2001-March/001198.html > Anyway, about your second point, we have the complete opposite philosophy: > run time checking is of lesser interest to us. Flexible, rich > introspection for tools and programmers to address the stated goals in the > PEP is our main focus. Not run time checking. In that case I wouldn't really support a new syntactic element. You could do it all in strings and it would be clearer to everyone that there are no real runtime implications (other than introspection). Heck, you could use XML documents with hyperlinks. :) I think that using the word interface for something that has no type-checking connotation will be confusing to those coming from every other language and OO system such as Java, COM, CORBA, XPCOM, etc. You'd be better off calling them "contracts" or "protocols". -- Take a recipe. Leave a recipe. Python Cookbook! http://www.activestate.com/pythoncookbook From Samuele Pedroni Thu Mar 15 22:12:49 2001 From: Samuele Pedroni (Samuele Pedroni) Date: Thu, 15 Mar 2001 23:12:49 +0100 (MET) Subject: [Types-sig] Interface PEP Message-ID: <200103152212.XAA10353@core.inf.ethz.ch> Hi. > because as far as I have understood: the semantic of the following definition > is that: > > class A implements I: > ... > > should enforce through a check that A offers all the signatures required by I. Sorry, I have reread the PEP, this is false, Could make sense but is not required by the PEP. > And conventionally expresses that A meet all of the implicit requirements > (not at signature level) that come with I. I have tried to understand both proposals, IMHO the tentative to force a merging between the two (which could have contact point and find a way to interact properly) is bringing just confusion. I repeat: I see the following inclusion relations: (Michel Pelletier) interfaces < protocols < domains ........._____________________ Paul Prescod "type" checking then there is the question how should __check__ and isImplementedBy interact wrt to runtime arg domain checking. and the possibility that domains could be used to specify parameters "types" in Michel Pelletier interfaces. regards, Samuele Pedroni. Note: in Objective-C interfaces are called protocols ;). Is just a matter of picking your names. From qrczak@knm.org.pl Thu Mar 15 21:57:31 2001 From: qrczak@knm.org.pl (Marcin 'Qrczak' Kowalczyk) Date: 15 Mar 2001 21:57:31 GMT Subject: [Types-sig] Type Annotations Progress report References: <3AB0EA75.554BF701@ActiveState.com> <057f01c0ad82$4017f700$87740918@CX781526B> Message-ID: Thu, 15 Mar 2001 12:00:49 -0700, Tim Hochberg pis= ze: > Also, I see that CheckInterface, if passed a tuple or a list as in > interface, checks each item in the checked object against each item > in the interface. IMHO checking that all elements of a list (an unspecified number) implement an interface is far more common than checking a particular list pattern. Lists are more or less homogeneous. When elements don't have the same role, a tuple is used. So if (Interf1, Interf2, Interf3) could play the role of an interface pattern for items, then [Interf] should mean that it's a sequence whose all items implement Interf. --=20 __("< Marcin Kowalczyk * qrczak@knm.org.pl http://qrczak.ids.net.pl/ \__/ ^^ SYGNATURA ZAST=CAPCZA QRCZAK From qrczak@knm.org.pl Thu Mar 15 22:09:00 2001 From: qrczak@knm.org.pl (Marcin 'Qrczak' Kowalczyk) Date: 15 Mar 2001 22:09:00 GMT Subject: [Types-sig] Type Annotations Progress report References: <3AB0EA75.554BF701@ActiveState.com> <057f01c0ad82$4017f700$87740918@CX781526B> Message-ID: Thu, 15 Mar 2001 12:00:49 -0700, Tim Hochberg pis= ze: > On the third hand, the above syntax puts a kink in my hope of using & > and | since: >=20 > val : (IString, IInteger) | (IFloat, IFloat) >=20 > won't work. Of course it will. Tuples will implement | and &. It's a type error if tuple items and the other argument of there operators are not interfaces:-) --=20 __("< Marcin Kowalczyk * qrczak@knm.org.pl http://qrczak.ids.net.pl/ \__/ ^^ SYGNATURA ZAST=CAPCZA QRCZAK From paulp@ActiveState.com Fri Mar 16 00:10:01 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Thu, 15 Mar 2001 16:10:01 -0800 Subject: [Types-sig] Type Annotations Progress report References: <3AB0EA75.554BF701@ActiveState.com> <057f01c0ad82$4017f700$87740918@CX781526B> Message-ID: <3AB159D9.75B551D4@ActiveState.com> Marcin 'Qrczak' Kowalczyk wrote: > >... > So if (Interf1, Interf2, Interf3) could play the role of an interface > pattern for items, then [Interf] should mean that it's a sequence > whose all items implement Interf. I would actually rather use the syntax [Interf, ...], but my concern with this comes back to implementation. I'll put the basic system out and let people talk about good models for handling the hard parameterizations. Once we've got a good runtime model with acceptable performance, we can put in the syntax for it. -- Take a recipe. Leave a recipe. Python Cookbook! http://www.activestate.com/pythoncookbook From pedroni@inf.ethz.ch Fri Mar 16 01:49:16 2001 From: pedroni@inf.ethz.ch (Samuele Pedroni) Date: Fri, 16 Mar 2001 02:49:16 +0100 Subject: [Types-sig] Interfaces API: a concrete contact point References: <3AB13CDD.96D1D4DA@ActiveState.com> Message-ID: <3AB1711C.8060904@inf.ethz.ch> Hi. If we really aim for having just interfaces (still not convinced about that ) IMHO a possible solution is to define their check this way: if the input to check against an interface IFace is an instance foo, we check if foo has an '__implements__' attribute in that case we test IFace.isImplementedBy(foo), otherwise we check if foo has all the required signatures :). Then eventually "class A implements IFace" should check at definition time if A has all the needed signatures, raising warnings for the missing signatures still allowing that way incremental coding. regards From barry@digicool.com Fri Mar 16 03:14:33 2001 From: barry@digicool.com (Barry A. Warsaw) Date: Thu, 15 Mar 2001 22:14:33 -0500 Subject: [Types-sig] Re: Type equivalence References: <15024.53139.238620.215002@anthem.wooz.org> Message-ID: <15025.34073.927004.230268@anthem.wooz.org> >>>>> "MK" == Marcin Kowalczyk writes: MK> This is strange. Why not just have true and false as canonical MK> truth values, without magic in comparisons, and keeping old MK> semantics of which values are considered true? Well if {1:2} is considered "true" and true is considered "true", then shouldn't true == {1:2}? Seems weird to me that it wouldn't be. MK> The biggest problem is that letting == < etc. return MK> false/true instead of 0/1 breaks compatibility a lot. I didn't say comparison operators should return the magic true or false values. Then again maybe it should. Yes, it breaks compatibility but wouldn't it be more type safe? -Barry From paulp@ActiveState.com Fri Mar 16 06:02:50 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Thu, 15 Mar 2001 22:02:50 -0800 Subject: [Types-sig] Interfaces API: a concrete contact point References: <3AB13CDD.96D1D4DA@ActiveState.com> <3AB1711C.8060904@inf.ethz.ch> Message-ID: <3AB1AC8A.5E814DFF@ActiveState.com> Samuele Pedroni wrote: > > Hi. > > If we really aim for having just interfaces (still not convinced about > that ) IMHO a possible solution is to define their check this way: > > if the input to check against an interface IFace is an instance foo, we > check if foo has an '__implements__' > attribute in that case we test IFace.isImplementedBy(foo), otherwise we > check if foo has all the required signatures :). That's exactly what my code already implements: # if the object CLAIMS to support an interface, we believe it if _findInterface(val, iface): return 1 # if the object LOOKS like it supports an interface, good 'nuff else: return iface.__check__(val) findInterface recursively looks for this interface or any of its base interfaces in the list of values > Then eventually > "class A implements IFace" should check at definition time if A has all > the needed signatures, > raising warnings for the missing signatures still allowing that way > incremental coding. Agreed. -- Take a recipe. Leave a recipe. Python Cookbook! http://www.activestate.com/pythoncookbook From tim.one@home.com Fri Mar 16 06:01:06 2001 From: tim.one@home.com (Tim Peters) Date: Fri, 16 Mar 2001 01:01:06 -0500 Subject: [Types-sig] Type Annotations Progress report In-Reply-To: <3AB0EA75.554BF701@ActiveState.com> Message-ID: [Paul Prescod] > Before I documented everything I wanted to do a real-world test. So I > added type annotations to the os module. I even added a new type that > was local to that module for handling P_WAIT, P_NOWAIT. I'd appreciate > if people would at least take a cursory peek at the os module, if not > the implementation. > > OS: www.prescod.net/pytypes/os.py > Implementation: http://www.prescod.net/pytypes/typecheck.py Paul, you know the rule: each time this SIG reincarnates, the first msg announcing an actual implementation kills it again <0.7 wink>. > ... > Remember that this is experimental, transitional, prototype, type > annotation system. The syntax is not ideal. Nevertheless, I think it is > quite simple, easy to use and immediately useful. Yup! Note that there are several misdeclarations, such as: def execl(file, *args): """execl(file, *args) Execute the executable file with argument list args, replacing the current process. """ __types__= {"file: Old directory name": IString, "args: New directory name": IString} __paramcheck__() execv(file, args) args should be a tuple of IStrings, right? mostly-a-type-system-exists-to-help-programmers-debug-type- declarations-ly y'rs - tim From paulp@ActiveState.com Fri Mar 16 06:19:11 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Thu, 15 Mar 2001 22:19:11 -0800 Subject: [Types-sig] Example param/rc declarations Message-ID: <3AB1B05F.A083A559@ActiveState.com> For those that don't have time to look at os.py, here are some example declarations: class _Environ(UserDict.UserDict): def __init__(self, environ): __types__= {"environ: Environment": IMapping } __paramcheck__() UserDict.UserDict.__init__(self) data = self.data for k, v in environ.items(): data[k.upper()] = v def __setitem__(self, key, item): __types__= {"self: Key to set": IString, "item: Value to insert": IString } __paramcheck__() putenv(key, item) self.data[key.upper()] = item def __getitem__(self, key): __types__= {"key: Key to get": IString, "_RETURNS: Value": IString } __paramcheck__() return __rccheck__(self.data[key.upper()]) def __delitem__(self, key): __types__= {"key: Key to delete": IString} __paramcheck__() del self.data[key.upper()] def has_key(self, key): __types__= {"self: Key": IString, "_RETURNS: key in dictionary": IStrictBoolean } __paramcheck__() return __rccheck__(self.data.has_key(key.upper())) Of course the syntax will be better when we can change Python... -- Take a recipe. Leave a recipe. Python Cookbook! http://www.activestate.com/pythoncookbook From paulp@ActiveState.com Fri Mar 16 06:19:42 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Thu, 15 Mar 2001 22:19:42 -0800 Subject: [Types-sig] Type declarations Message-ID: <3AB1B07E.6EA85CF5@ActiveState.com> Here are some type declarations for the types used in the example: def _ismapping(obj): if type(obj)==types.InstanceType: return (hasattr(obj, "__getitem__") and hasattr(obj, "items")) else: return operator.isMappingType(obj) IMapping = Interface(name = "IMapping", doc = "Mapping from keys to values", testfunc = _ismapping) IString=Interface(name = "IString", doc = "Unicode or 8-bit string", testfunc = lambda obj: type(obj) in (types.StringType, types.UnicodeType)) IStrictBoolean = Interface(name = "IStrictBoolean", doc = "0 or 1 value", testfunc = lambda obj: obj in (0, 1)) -- Take a recipe. Leave a recipe. Python Cookbook! http://www.activestate.com/pythoncookbook From paulp@ActiveState.com Fri Mar 16 06:25:47 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Thu, 15 Mar 2001 22:25:47 -0800 Subject: [Types-sig] Type Annotations Progress report References: Message-ID: <3AB1B1EB.148C85B9@ActiveState.com> Tim Peters wrote: > > ... > > Paul, you know the rule: each time this SIG reincarnates, the first msg > announcing an actual implementation kills it again <0.7 wink>. It has been kind of quiet today. :) > ... > args should be a tuple of IStrings, right? Yeah, adding type declarations at 4:00 in the morning is not the kind of thing that holds my attention. > mostly-a-type-system-exists-to-help-programmers-debug-type- > declarations-ly y'rs - tim Static typing fans will point out that ML wouldn't have let my errors past the compiler. I would counter by pointing out that this file passed the standard test suite so the flaw is really with the lack of coverage of the suite and not with my type system per se. :) I *would* counter with that but I don't want to get stuck with the job of writing test_os.py. It would be almost as exciting as adding the type declarations. :) -- Take a recipe. Leave a recipe. Python Cookbook! http://www.activestate.com/pythoncookbook From qrczak@knm.org.pl Fri Mar 16 06:58:11 2001 From: qrczak@knm.org.pl (Marcin 'Qrczak' Kowalczyk) Date: 16 Mar 2001 06:58:11 GMT Subject: [Types-sig] Re: Type equivalence References: <15024.53139.238620.215002@anthem.wooz.org> <15025.34073.927004.230268@anthem.wooz.org> Message-ID: Thu, 15 Mar 2001 22:14:33 -0500, Barry A. Warsaw pis= ze: > Well if {1:2} is considered "true" and true is considered "true", then > shouldn't true =3D=3D {1:2}? [3] is also considered "true", but {1:2} !=3D [3]. --=20 __("< Marcin Kowalczyk * qrczak@knm.org.pl http://qrczak.ids.net.pl/ \__/ ^^ SYGNATURA ZAST=CAPCZA QRCZAK From Samuele Pedroni Fri Mar 16 12:21:22 2001 From: Samuele Pedroni (Samuele Pedroni) Date: Fri, 16 Mar 2001 13:21:22 +0100 (MET) Subject: [Types-sig] Interfaces API: a concrete contact point: more subtlities Message-ID: <200103161221.NAA00614@core.inf.ethz.ch> Hi. Sorry for insisting Paul Prescod wrote: > > Samuele Pedroni wrote: > > > > Hi. > > > > If we really aim for having just interfaces (still not convinced about > > that ) IMHO a possible solution is to define their check this way: > > > > if the input to check against an interface IFace is an instance foo, we > > check if foo has an '__implements__' > > attribute in that case we test IFace.isImplementedBy(foo), otherwise we > > check if foo has all the required signatures :). > > That's exactly what my code already implements: I intended something slightly different. (A) > > # if the object CLAIMS to support an interface, we believe it > if _findInterface(val, iface): > return 1 Namely: if hasattr(val,'__interfaces__'): # or Michel Pelletier __implements__ return _findInterface(val,iface) else: ... So to say to be on the safe side. Clearly if one want to have something which implements for example __getitem__ and some "true" interfaces, he will have to declare expliclitly ISequence too: class A implements IFoo,IBat,ISequence: ... def __getitem__(self,ind): ... Not that nice ;). (B) > > # if the object LOOKS like it supports an interface, good 'nuff > else: > return iface.__check__(val) > Clarifying this I'm more conviced about my initial opinion that this merging is just confusing: (A) is the right kind of "type checking" for the for-runtime-reflection-&-eventually-for-design-decisions-enforcing interfaces of Michel Pelletier proposal, which (I imagine) will carry a lot of implicit requirements (B) express the plain-old python approach to protocols, it is mostly needed for "type checking" the std lib, and the checked things would be special __foo__ methods or having simple things like a 'read' or 'readline', but checking anything too much complex this way is expensive and does not make (IMHO) that much sense. I repeat sometimes one would prefer doing "type checking" for an interface as in (A), sometimes like in (B) (especially for the std lib), they are different approaches, and there may be a need to distinguish them and spell this out. OTOH: for defining protocols ("interfaces") for the (B)-kind of test the full featured interface syntax is not really necessary: mostly they will be predefined (the common protocols: sequence etc) or expressed through somekind of anonymous protocol/interface/domain construction (as proposed by Tim Hochberg). regards, Samuele Pedroni From paulp@ActiveState.com Fri Mar 16 15:08:10 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Fri, 16 Mar 2001 07:08:10 -0800 Subject: [Types-sig] Re: Type equivalence References: <15024.53139.238620.215002@anthem.wooz.org> <15025.34073.927004.230268@anthem.wooz.org> Message-ID: <3AB22C5A.ADDBDECB@ActiveState.com> It would not be inconsistent for Python's boolean-style operators to work on multiple types and yet for Python to have a first-class boolean type. C++ is one example of a language with this behavior. Nobody is particularly confused by it. One way to make it all feel object-oriented and orthogonal is to say that __nonzero__ is a cast-to-boolean operator. -- Take a recipe. Leave a recipe. Python Cookbook! http://www.activestate.com/pythoncookbook From paulp@ActiveState.com Fri Mar 16 15:53:12 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Fri, 16 Mar 2001 07:53:12 -0800 Subject: [Types-sig] Interfaces API: a concrete contact point: more subtlities References: <200103161221.NAA00614@core.inf.ethz.ch> Message-ID: <3AB236E8.70892896@ActiveState.com> Samuele Pedroni wrote: > > > My code: > > # if the object CLAIMS to support an interface, we believe it > > if _findInterface(val, iface): > > return 1 > Your code: > if hasattr(val,'__interfaces__'): # or Michel Pelletier __implements__ > return _findInterface(val,iface) > else: > ... Why would we return if it doesn't support the interface? I think we should go on to check conformance to the interface, even though the object doesn't DECLARE conformance to it. Python users have expressed that it is very important to them that objects that look like they conform should pass the type-checker. > ... > (A) is the right kind of "type checking" for the > for-runtime-reflection-&-eventually-for-design-decisions-enforcing > interfaces of Michel Pelletier proposal, which (I imagine) will > carry a lot of implicit requirements > > (B) express the plain-old python approach to protocols, it is mostly > needed for "type checking" the std lib, and the checked things > would be special __foo__ methods or having simple things like a 'read' > or 'readline', but checking anything too much complex this way is expensive > and does not make (IMHO) that much sense. In my opinion, that is the beauty of putting them together. If a user claims to support the SocketServer interface, we trust them and they get a very efficient pointer check. Maybe we'll even get to the point where we check conformance to interfaces at class definition time so that trusting them is relatively safe. On the other hand, if a user just has an object that is eqiuvalent to a SocketServer but they haven't actually tagged it with the interface, that will be relatively safe also. Anyhow, I don't have to merge Michel's syntax explicitly. Maybe I can convince you that in the closed world of my proposal, the proposed algorithm makes sense and then extension to Michel's world can proceed later. We will allow the definition of relative expensive checks. Even the checks for certain methods will be relative expensive. Therefore it makes sense to allow objects to say: "I know that I conform to this protocol. Don't bother to do the expensive check." The simplest way to do this sort of thing is pointer equality with the protocol object. -- Take a recipe. Leave a recipe. Python Cookbook! http://www.activestate.com/pythoncookbook From Samuele Pedroni Fri Mar 16 18:13:47 2001 From: Samuele Pedroni (Samuele Pedroni) Date: Fri, 16 Mar 2001 19:13:47 +0100 (MET) Subject: [Types-sig] Interfaces API: a concrete contact point: more subtlities Message-ID: <200103161813.TAA10501@core.inf.ethz.ch> Hi. I will not insist further. This is my last try on the topic. [Paul Prescod] > > Samuele Pedroni wrote: > > > > > > My code: > > > # if the object CLAIMS to support an interface, we believe it > > > if _findInterface(val, iface): > > > return 1 > > > Your code: > > if hasattr(val,'__interfaces__'): # or Michel Pelletier __implements__ > > return _findInterface(val,iface) > > else: > > ... > > Why would we return if it doesn't support the interface? I think we > should go on to check conformance to the interface, even though the > object doesn't DECLARE conformance to it. Python users have expressed > that it is very important to them that objects that look like they > conform should pass the type-checker. If you reread the archive, at least in this discussion, I was the first pointing out that, and speaking about "selectors" and similarity with Smalltalk ;). http://mail.python.org/pipermail/types-sig/2001-March/001153.html Yes that's important, especially if one want to put checks everywhere in the stdlib, and in that case just the name of the method is saying a lot (e.g. __getitem__) on the programmer intention like it is in Smalltalk tradition. This is not so true for file-like methods but if we are to restrictive we will make people unhappy. But the interfaces as proposed by Michel Pelletier are there to express complicated apllicative logic. > > > ... > > (A) is the right kind of "type checking" for the > > for-runtime-reflection-&-eventually-for-design-decisions-enforcing > > interfaces of Michel Pelletier proposal, which (I imagine) will > > carry a lot of implicit requirements > > > > (B) express the plain-old python approach to protocols, it is mostly > > needed for "type checking" the std lib, and the checked things > > would be special __foo__ methods or having simple things like a 'read' > > or 'readline', but checking anything too much complex this way is expensive > > and does not make (IMHO) that much sense. > > In my opinion, that is the beauty of putting them together. If a user > claims to support the SocketServer interface, we trust them and they get > a very efficient pointer check. Maybe we'll even get to the point where > we check conformance to interfaces at class definition time so that > trusting them is relatively safe. AFAIK for very complicated things the actual way of life of python is to furnish abstract classes, a SocketServer typecheck will be a check against being an instance of a SocketServer class. This may change once we have Michel's interfaces but if you're writing something heavyweight putting an implements assertion does not cost that much and could even make sense and in any case is Michel Pelletier proposed philosophy, he want people doing that ;). IMHO mixing the tradition of well-established at language or lib level tiny protocols: e.g. the special __magic__ methods and file-like objects, and the heavyweight new interfaces of Michel Pelletier with this kind of default checking is just apparently clean. With an imaginative and hyper-tentative syntax I think it would be better to have full control about what we want to check! protocol PReadable # but we won't need that, better anonymous protocol # constructors as Tim Hochberg proposed def read(size) interface IStarTrekCompilerVisitor ... interface IWorkerThread: def input(in: PReadble) # check if 'in' has 'read' def star_ast_visit(visitor: IStarTrekCompilerVisistor) # check isImplementedBy and eventually def schedule(thread: IWorkerThread.loose) # check isImplementedBy or signatures ... That's all from my part. regards. PS: 1) the idea about just checking for methods is mostly a workaround to cope with actual tradition and to save people from having to put implements assertion in their old code, which would be even unnecessary because if a class implements a special __spell__ method is very likely at least trying to do the right thing. 2) Wanting to make typecheks is already breaking the similarity with Smalltalk ;). 3) checking for signatures looking at Smalltalk way-of-life would require (in principle) to check even for arg names, in Smalltalk they are part of the selectors. But to have uniform args naming is not in python tradition :-(. We are just trying to find a compromise, it should be a little bit ugly. From Samuele Pedroni Fri Mar 16 18:26:16 2001 From: Samuele Pedroni (Samuele Pedroni) Date: Fri, 16 Mar 2001 19:26:16 +0100 (MET) Subject: [Types-sig] the very last thing Message-ID: <200103161826.TAA10746@core.inf.ethz.ch> Here's a concrete example why it makes sense to distinguish the various kinds of checking: once you have interfaces it is typical (so it is in java AFAIK) to use very inexpressive names for the methods, something along the way of: interface ObjectDocReader: def read() a typechecking that in this case is just happy about finding a 'read' method will be easely defeated and your program will deliver you the old weird errors. Better to ask people to put an implements assertion in their new code, in this case. regards. From itamar@maxnm.com Fri Mar 16 22:23:25 2001 From: itamar@maxnm.com (itamar@maxnm.com) Date: Sat, 17 Mar 2001 00:23:25 +0200 (IST) Subject: [Types-sig] Pre- And post-conditions for Interfaces Message-ID: Hi, I've written a sample implementation of optionally enforcable pre and post conditions, based off Michel's interface code. Integration with the documentation generator would make them even more useful. Please check them out and mail me 9I'm not yet subscribed) suggestions or rants. http://itamarst.org/downloads/prepost-interfaces-0.1.tar.gz -- Itamar Shtull-Trauring, http://itamarst.org From michel@digicool.com Sat Mar 17 02:36:10 2001 From: michel@digicool.com (Michel Pelletier) Date: Fri, 16 Mar 2001 18:36:10 -0800 (PST) Subject: [Types-sig] Pre- And post-conditions for Interfaces In-Reply-To: Message-ID: Itamar, I haven't had a really good chance to take a look at your code. Is there any kind of prose description you could give me? There's a good bit of code there and it's hard for me to follow exactly its intent. If you can give me a clear enough description I'll add a section to the PEP. -Michel On Sat, 17 Mar 2001 itamar@maxnm.com wrote: > Hi, > > I've written a sample implementation of optionally enforcable pre and post > conditions, based off Michel's interface code. Integration with the > documentation generator would make them even more useful. Please check > them out and mail me 9I'm not yet subscribed) suggestions or rants. > > http://itamarst.org/downloads/prepost-interfaces-0.1.tar.gz > > -- > Itamar Shtull-Trauring, http://itamarst.org > > > _______________________________________________ > Types-SIG mailing list > Types-SIG@python.org > http://mail.python.org/mailman/listinfo/types-sig > From qrczak@knm.org.pl Sat Mar 17 13:04:09 2001 From: qrczak@knm.org.pl (Marcin 'Qrczak' Kowalczyk) Date: 17 Mar 2001 13:04:09 GMT Subject: [Types-sig] Interface PEP References: <3AAFCBDF.39A0735D@home.se> <3AB00D5A.453E7BCF@home.se> Message-ID: Thu, 15 Mar 2001 01:31:22 +0100, Sverker Nilsson pis= ze: > What not only many but perhaps all people would agree on, including > me, is that the builtin types or classes should be unified with > user-defined classes in this way: So that user-defined classes can > inherit from the classes that the objects with builtin types have. I agree. > That doesn't necessary mean unifying types with classes. It could just > as well mean that we defined the classes of the builtin objects. Why not to unify then? I understand that there are technical reasons, like the speed resulting from avoiding looking up operations by name, or the fact that if calling an object is equivalent to calling its __call__ method, then x(4) is x.__call__(4) which is x.__call__.__call__(4) which is x.__call__.__call__.__call__(4) etc., so we must break the conceptual madness and say that calling a builtin function object is not further reduced to calling its method but it has the callable capability built in. But from the point of users of the language there is no reason why files should be objects of FileType and not objects of InstanceType with class File. It's an artifact of the implementation. > Why would it be cleaner to unify type and class, given the many > variants that exist, that you describe (some of) yourself below? Because concepts for which many other languages use terms "type" and "class" are more different than concepts for which Python uses these terms (except C++, but let's not follow this crazy language). These words have different meanings in those languages and whether types and classes can be unified in Python is independent from whether they can be unified in Haskell. It's just a terminology clash. > below you say that Eiffell had problems with having to analyze too > much to see if a class would yield a subtype. >=20 > But here you seem to say that type is separate from class, and it > describes an interface. Well, the term "type" is used when talking about Eiffel but the language syntax has no "type declarations" which bind types to names or the like. ARRAY[INTEGER] is a type, because ARRAY is a generic class and INTEGER is a type. INTEGER is a type because INTEGER is a class. That is, for any non-generic class (a module which defines a set of features, inheritance relations etc.) there exists a type based on it (a sequence of characters that you can write e.g. when declaring an argument of a method, which denotes that this method accepts particular objects here, and can use features defined by the class on which the type is based). In a given context you always talk about a type or about a class. It doesn't make sense to say "this may be either a type or a class". Types are based on classes and classes yield types, but the intersection of set of types and the set of classes is empty - these are separate concepts. > So the problems you say they had, seem to be the same whether they > called the type a type or interface. It's because they try to unify subtyping interfaces with inheriting implementation (like C++ and Java do). Eiffel allows viewing an object of a subclass as an object of a superclass, including assignment to variables declared as holding objects of the superclass. This is subtyping. Eiffel also allows covariant changes of method types (replacing declared types of method arguments with subtypes) and hiding features. This is inheritance and there is nothing wrong with it. What is wrong is when these two cases are combined, and a type is used both to describe an interface (to pass objects of various types under a parameter or attribute declared with that type) and to inherit implementation (to let some classes include its features by default and state only the differences) when the latter uses covariant changes or feature hiding. This is unsound: the type system would not catch some incompatible feature calls. There are additional factors which makes it worse (maybe both are aspects of the same problem): 1. Eiffel doesn't allow to have functions parametrized by types, only classes parametrized by types, so it uses subtyping (a method accepting two objects of type COMPARABLE) to express genericity (a method should accept two object os any type which is COMPARABLE). Haskell solves this case correctly by making interfaces (like COMPARABLE, called Ord there) a different concept than types, and applying interface constraints to types of parameters instead of parameters themselves. 2. Eiffel assumes that all arguments of generic types are covariant, i.e. that ARRAY[INTEGER] is a subtype of ARRAY[COMPARABLE] only because INTEGER is a subtype of COMPARABLE. But this is not true in general, in particular for mutable arrays. Array item assignment is contravariant by nature (if you want to put an INTEGER in an array, you would accept a command which promises to take a COMPARABLE - not vice versa). Other array operations are covariant (if you want to extract a COMPARABLE, you will accept a function which promises to extract an INTEGER). This makes the array type invariant. I don't remember how Eiffel handles the problem. Java has the same problem and throws a runtime exception if an array of B is coerced to an array of B's superclass A and a wrong object is assigned to it. Haskell doesn't have the problem because it doesn't use subtyping to express genericity. OCaml sometimes uses subtyping. Until recently it treated all generic types as invariant (a list of As is not a subtype of a list of Bs no matter how A and B are related, except when A =3D B). It doesn't bite much because subtyping is not used much. Recently (version 3.01) it added variance inference and explicit variance annotations, which allows to coerce a list of As to a list of Bs if A is a subtype of B, because the list type is covariant in its parameter. Arrays are invariant because they are mutable. The function type is contravariant on the argument type and covariant on the result type. > Haskell is arguably one of the more sofisticated. Still they had > problems defining general container classes, at least before they > introduced dual inheritance. Indeed container classes have some problems for the Haskell's type system if they are to be generic enough (i.e. when some containers are fully generic and others work only on element types conforming to some interfaces or being a concrete type). It works quite well if sequences, sets/bags, dictionaries and arrays are treated separately. Chris Okasaki has done this (Edison library). It doesn't work well if one wants to use the same method for operations on different kinds of collections with different enough signatures (e.g. if the same method is used for adding an element to a set and a pair to a dictionary) - it gets messy and requires further language extensions. > (I don't know if that's there now officially, I remember it was > controversal.) I assume that you refer to multiparametric classes. They are indeed not in Haskell 98, and there is no other official standard. They are supported by 2 out of 4 implementations (one of the implementations where they are not supported is dead, the other is alive). > And I read some paper that showed how impossible it was to define > a natural join... in some context. I don't know what is natural join. > I'd be happy to use OCaml for its superb speed, have looked just a > little in it, but may be held back afraid of future refactoring > problems due to the static type system. >=20 > BTW, does it have any dynamic types at all? It has polymorphic variants (like algebraic types, but with variants not belonging to particular types), which provide a larger flexibility than traditional algebraic types while still statically ensure type correctness. It has Obj.magic which coerces an object of any type to any other type, if you want to skip the type system. [...] > It's just a special case. Extending our knowledge could be made by > having the class type be something more specific than InstanceType. >=20 > Possibly the type could be dynamically generated and contain a pointer > to the object, to allow for generality in checking that the methods > are compatible with some other type (interface). Looks very weird for me. Python's types in fact reflect the internal representation, with some builtin cases and one generic case. I don't see how this mechanism could help with interfaces. Even if it was technically doable, it's wrong. An object doesn't have a single interface. For two independent interfaces the fact whether an object implements one of them is unrelated to the fact whether it implements the other. There is no function object->interface but at most a relation over pairs. > > I assume that we use the term "type" as currently defined in Python, > > and don't change that. >=20 > Now it just means the types defined in types.py. Right? > Are you saying that no more types should be added, except by > new builtin objects? No, I'm saying that we shouldn't change which concept is called "type" (and "class" too) in Python, at least until types are classes are unified. We may introduce new concepts, "interface", "protocol", whatever, or change properties of types and classes a bit, but radical terminology changes would be confusing. > > ListType describes an implementation, i.e. how an object was > > constructed. There is a particular object layout and particular > > methods. >=20 > When something has type ListType you know its particular methods.=20 > So why doesnt ListType define an interface too? Because an interface rarely says anything about the representation. Other types supporting the same interface should be generally accepted where lists are accepted. ListType says too much about an object. Python has exceptions to this. Many builtin types require other builtin types, e.g. __dict__ of a builtin class object must have the builtin dictionary type. It's slowly shifting to allow more generality, but... ...concrete types must be finally used somewhere. You can express an abstract interface of an integer, but you can't express every abstract interface in terms of other abstract interfaces. Finally some real work must be done besides message passing. If you try to express pure Smalltalk model in Haskell: newtype Object =3D O (String -> [Object] -> IO Object) then you can't do anything useful unless you encode data in method names, because the only thing you can do with an object is to send it a message, so you can't know which damn integer number an object represents to implement arithmetic or to index a sequence. Some objects or some messages must be special. There is a balance between concrete and abstractness. It's impossible to be completely abstract, and should not be completely concrete because it's very painful to use (as in C and Pascal). > It also defines a particular object layout as you say. I'd say that > should be considered a part of the interface, too. Most of the time, no. > > SequenceType describes an interface, i.e. how an object can be used. >=20 > How an object can be used, isn't that exactly also what ListType > describes? ListType describes too much. BTW, functional languages, especially lazy ones, better express abstract interfaces in terms of concrete types than traditional statically typed OO languages. In "functional" function closures are important here, not the immutability of data. A function closure can have many implementations. As long as they use the same argument and result types, they are expressed in the same function type. Contrast it with virtual methods in C++ or Java, where different implementations of an abstract class are considered separate types which can only be coerced to the common supertype. Haskell doesn't have subtyping, but many OO style examples can be straightforwardly converted to closures. Laziness implies that a String object can cause arbitrary computation to be performed while it is examined. Haskell doesn't use iterators because lazy lists are enough. A list of ints has the same type as all lists of ints even if evaluation of this particular list traverses a tree in preorder. These two aspects are related. Laziness can be expressed in a language without builtin laziness using function closures. Lazy values are similar to nullary functions. > > A basic misunderstanding of some languages is confusing subtyping > > (the ability to use one type if another is required, i.e. a relation > > on interfaces) and inheritance (basing a class definition on another > > class, i.e. reusing the implementation). >=20 > So if I am understanding you correctly: >=20 > The problem is you get a type (=3D class) that is based on a particular > implementation? And not being able to control the interface separately > from the implementation? Here I had in mind Eiffel's problems. > Yes in C++ there is no interface concept separate from classes, right? Right. And thus C++ doesn't allow to change method signatures in subclasses (except covariant change of the return type if it's a pointer or reference; this feature is not implemented by all compilers). Fortunately in C++ it doesn't bite much because genericity is not always expressed as subclassing, but by templates. Parametrization by types is powerful and works well with static typing. > > But you try to unify types / classes with interfaces, which > > doesn't work well. >=20 > No I wasnt trying to unify to 1 concept. Just to 2 concepts: types > and classes. This is very strange. Python's types and classes are used for the same purpose: to implement objects, define how method calls are dispatched, and finally get to object's data assuming it has the right type or class (no matter if it's PyInt_AsLong in a C code or a private attribute reference in pure Python's code). Interfaces are used for another purpose: to define what kinds of objects are expected in which places, and to formalize what is needed to use the same general functionality on different kinds of objects. An object has one type and one class (even if can be considered to implicitly have its superclasses too), but implements many interfaces independently of each other. Types define the representation. Interfaces define the usage. Classes are somewhat between. In dynamically typed languages they are much closer to types, because usage is not checked wrt. declared classes - in Python subclassing is only used to inherit behavior, not to create subtypes. Sorry, unifying types and interfaces while leaving classes alone is complete nonsense. --=20 __("< Marcin Kowalczyk * qrczak@knm.org.pl http://qrczak.ids.net.pl/ \__/ ^^ SYGNATURA ZAST=CAPCZA QRCZAK From qrczak@knm.org.pl Sat Mar 17 13:49:13 2001 From: qrczak@knm.org.pl (Marcin 'Qrczak' Kowalczyk) Date: 17 Mar 2001 13:49:13 GMT Subject: [Types-sig] Another cut at Sequence declarations References: <3AAE220A.DC92ABED@ActiveState.com> <036401c0acdc$0617a790$87740918@CX781526B> Message-ID: Wed, 14 Mar 2001 16:10:56 -0700, Tim Hochberg pis= ze: > I defined three protocols: Iterable, BasicSequence and Sequence. > Iterable is something you can iterate on. Currently that means it > supports getitem, but this will be expanded to include whatever > the iter-sig comes up with. BasicSequence supports getitem and > len. This is the minimum that you need to construct an immutable > sequence (e.g., a tuple). What about lazily computed or infinite sequences, when you should not request the length in advance, or even ever? Fortunately iterators discussed on iter-sig don't work in terms of indexing and length, so they are compatible with lazy lists (e.g. reading lines from a file). > Finally, there is a class SequenceWrapper that creates a Sequence out o= f > a BasicSequence. It's used as "aSequence=3DSequenceWrapper(aBasicSequen= ce)" > I'm not sure how necessary this is since "tuple(aBasicSequence)" would > work as well or better in most situations. It won't work for lazy lists. --=20 __("< Marcin Kowalczyk * qrczak@knm.org.pl http://qrczak.ids.net.pl/ \__/ ^^ SYGNATURA ZAST=CAPCZA QRCZAK From robin.thomas@starmedia.net Sat Mar 17 19:45:17 2001 From: robin.thomas@starmedia.net (Robin Thomas) Date: Sat, 17 Mar 2001 14:45:17 -0500 Subject: [Types-sig] Proposal: "abstract" type checking Message-ID: <4.3.1.2.20010314213700.00d389c0@exchange.starmedia.net> I'm not knowledgeable enough to weigh in on static typing, the type/class dichotomy in CPython, etc. But I would like to propose the following change to (C)Python. If this proposal is redundant, please tell me. FWIW, I did research of existing stuff before posting this proposal. "ABSTRACT" TYPE CHECKING When you do this in Python: >>> type(obj) the built-in function type() returns a type object, which can be compared with other type objects (CPython compares type objects by object identity). As of 1.5.2 (?), the built-in function isinstance() accepts, as a convenience, a type object as its second argument, such that: >>> import types >>> isinstance(2, types.IntType) 1 I often find myself doing the following in Python. This is a synthetic example, not production code: def myfunc(x): t = type(x) if t in (ListType, TupleType): do_this_for_a_list_or_tuple(x) elif isinstance(x, DictType): do_this_for_a_dict(x) else: raise TypeError, "x must be list, tuple, or dict, you passed: %s" % repr(x) When I do this, I often don't care whether x really is a concrete list, tuple, or dictionary object. I only care that it supports some or all of the sequence or mapping protocols. The above works well, until I find myself wanting to define a dictionary-like class that will print debugging info, generate events, or something else whenever certain operations are performed on its instances: # abridged example class MyDict(UserDict.UserDict): def __setitem__(self, item, value): do_something_first(item, value) UserDict.UserDict.__setitem__(self, item, value) Now I cannot pass instances of MyDict to myfunc() without it raising a TypeError. At this point, I consider the following solutions: 1) Don't do run-time type checking in myfunc(). Just let the code blow up when it tries to operate on the object in a way it doesn't support. This would be acceptable if myfunc merely did type-checking as a "sanity check" on arguments. However, myfunc() also makes decisions based on the type of the argument. I also am distributing myfunc() to less experienced Python users. I don't want them to see AttributeError: has_key or something similar as their error, especially from a frame way down in the stack from myfunc(). I want them to see TypeError: x must be a list, tuple, or dictionary, you passed: 42 from the frame associated with myfunc(). This kind of "educational" error reporting is very important to my application and to our development efforts. 2) Don't check argument types; use hasattr() on the argument object to see if it supports the right protocols/operations. Nice try, but some types (like tuples) don't have methods or other meaningful attributes. Some of the built-in functions help, like len(), but in myfunc() I couldn't decide whether to use x as a list/tuple or a dict because all three object types have length. 3) Replace the built-in functions type() and isinstance() to allow an object to intercept a type check and claim to be of a type different than its concrete type. This works very, very well. I don't have to change any of my existing code, unless some of my Python code really demands that an object be of a certain concrete type. (I haven't found this case in real life yet,and if I did, the worst that could happen is that Python would generate a traceback, which is just fine and better than what happens with the previous alternatives. Working code is below. # abstract_type.py import __builtin__, types __type_concrete = __builtin__.type __isinstance_concrete = __builtin__.isinstance # this should probably be implemented as a builtin type, # and its concrete type listed in the types module # (but not __builtin__). class TypeDelegate: def __init__(self, realtype, dispatch): if __type_concrete(realtype) != types.TypeType: raise TypeError, "realtype must be type object" if __type_concrete(dispatch) not in (types.MethodType, types.FunctionType): raise TypeError, "dispatch must be method or function" self.realtype = realtype self.dispatch = dispatch def __cmp__(self, other): return self.realtype == other or self.dispatch(other) __rcmp__ = __cmp__ def type(obj, concrete=0): t = __type_concrete(obj) if concrete: return t # only concrete instances may implement # a special abstract type-checking method; # the exception approach can be replaced # but is very concise elif t == types.InstanceType: try: return TypeDelegate(types.InstanceType, t.__type__) except: pass return t def isinstance(obj, classobj, concrete=0): result = __isinstance_concrete(obj, classobj) if concrete: return result # abstract type check of classobj important; # used-defined "type objects" need to be supported if type(classobj) == types.TypeType: return type(obj) == classobj return result __builtin__.type = type __builtin__.isinstance = isinstance # end of abstract_type.py NOTES AND PROPAGANDA - Instances which emulate types will pass run-time type checking tests done with type() and the type-checking behavior of isinstance(). - You can create your own "type objects" to represent certain sets of functionality: subsets of object protocols (e.g. SliceableType, ReadOnlyDictType), : from types import * # maybe this too could be a builtin type, # but it doesn't have to be class Type: def __init__(self, types=None): if types: self.types = types else: self.types = None def __type__(self, type): return type == TypeType def __cmp__(self, other): t = self.types if t: return other in t else: return self is other NonStringlikeSequenceType = Type( (TupleType, ListType) ) SliceableType = Type() CanCallMethodFooType = Type() PersistableType = Type() - No code breakage: type(obj) still works. Anybody who really needs to know the *concrete* type of an object from Python can do type(obj, 1). Whether checking concrete type *in Python code* is a common case or (as I suspect) a very rare case is up for discussion. - Concrete type-checking at the C layer is completely unaffected. - Performance cost seems negligible. Your findings and opinions on this issue are welcome. - A logical extension of this proposal is to add a type-checking function at the abstract object layer, either PyObject *PyObject_GetType(PyObject *o) or int PyObject_CheckType(PyObject *o, PyObject *t). Abstract type comparison is really just an object comparison with a special, conventional meaning. - This proposal makes no strident claims regarding the Right Way to do "interfaces", resolve the class/type dichotomy, emulate types, or anything else. It doesn't just fail to address the class/type dichotomy; it actually respects it. It merely provides a standard hook for emulating object typing in an abstract sense, just as all of the special __methods__ allow concrete instance objects to assert emulation of numbers, sequences, and various object operations in an abstract way. - Individuals may explore different typing strategies without modifying or extending Python itself, and can put that code in production environments. Then they can *distribute* that code to others, and the new built-in abstract typing provides a standard way to conform to type-checking conventions. If you disagree with others about how typing can/should be used - Optionally, the old type and isinstance functions may be preserved in the builtin module under different, "preserved" names, a la __stdout__ for stdout in the sys module. (Allows brute-force backward compatibility in the rare case that your Python code is full of concrete type checks.) - New or changed code is very minimal. The full implementation adds two functions to bltinmodule.c (the replacements for type and isinstance), and one (or two) functions to Objects/abstract.c (for the abstract type checking functions for the C API). - Another way to bypass the "abstract typing hook" is to use the "is" operator for type object comparison. "type(obj) is types.DictType" effectively thwarts any attempts at abstract typing. One sticky example is "type(obj) is types.InstanceType" when obj is an instance with a __type__ method. Ack! Your feedback on this issue is greatly appreciated! - It seems like the idiom "type(obj) == some_type" is deprecated. That's why isinstance supports type checking, why the abstract object layer has PySequence_Check et al. I get the impression that type checking is encouraged as an operation on two objects, object and typeobject. If that's true, the above changes to type() and the TypeDelegate behavior are for backward compatibility with the idiom "type(obj) == some_type". -- Robin Thomas Engineering StarMedia Network, Inc. robin.thomas@starmedia.net From michel@digicool.com Sat Mar 17 22:51:49 2001 From: michel@digicool.com (Michel Pelletier) Date: Sat, 17 Mar 2001 14:51:49 -0800 (PST) Subject: [Types-sig] Proposal: "abstract" type checking In-Reply-To: <4.3.1.2.20010314213700.00d389c0@exchange.starmedia.net> Message-ID: Robin, You should definately sketch this up in a PEP. -Michel On Sat, 17 Mar 2001, Robin Thomas wrote: > I'm not knowledgeable enough to weigh in on static typing, the type/class > dichotomy in CPython, etc. But I would like to propose the following change > to (C)Python. If this proposal is redundant, please tell me. FWIW, I did > research of existing stuff before posting this proposal. -Michel From robin.thomas@starmedia.net Sun Mar 18 01:48:19 2001 From: robin.thomas@starmedia.net (Robin Thomas) Date: Sat, 17 Mar 2001 20:48:19 -0500 Subject: [Types-sig] Proposal: "abstract" type checking In-Reply-To: References: <4.3.1.2.20010314213700.00d389c0@exchange.starmedia.net> Message-ID: <4.3.1.2.20010317204345.00d6d6f0@exchange.starmedia.net> At 02:51 PM 3/17/01 -0800, Michel Pelletier wrote: >Robin, > >You should definately sketch this up in a PEP. I plan to. Do anyone have any feedback before I do so? -- Robin Thomas Engineering StarMedia Network, Inc. robin.thomas@starmedia.net From paulp@ActiveState.com Sun Mar 18 02:18:40 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Sat, 17 Mar 2001 18:18:40 -0800 Subject: [Types-sig] Proposal: "abstract" type checking References: <4.3.1.2.20010314213700.00d389c0@exchange.starmedia.net> <4.3.1.2.20010317204345.00d6d6f0@exchange.starmedia.net> Message-ID: <3AB41B00.EB25D996@ActiveState.com> Robin Thomas wrote: > > ... > This works very, very well. I don't have to change any of my existing code, > unless some of my Python code really demands that an object be of a certain > concrete type. (I haven't found this case in real life yet,and if I did, > the worst that could happen is that Python would generate a traceback, > which is just fine and better than what happens with the previous > alternatives. I find it somewhat uncomfortable to have objects masquerading as other objects. For one thing it means that you must implement each and every method implemented by the built-in types to be "safe". For another thing it may break existing code. Also, if I have a sequence, can I decide whether to emulate both list AND tuple? Can I also emulate a file if I happen to have file-like methods? I think we should move forward to a world where abstract protocols/interfaces/... are explicit (as they are in the C-world) rather than trying to pretend that ListType and TupleType *are* abstract types. I think it confuses things further. -- Take a recipe. Leave a recipe. Python Cookbook! http://www.activestate.com/pythoncookbook From cce@clarkevans.com Mon Mar 19 02:43:38 2001 From: cce@clarkevans.com (Clark C. Evans) Date: Sun, 18 Mar 2001 21:43:38 -0500 (EST) Subject: [Types-sig] Far from Imperfect Summary In-Reply-To: Message-ID: http://www.zope.org/Members/michel/types-sig/TreasureTrove An overall status summary: There are two (perhaps three) PEPs being created, with Paul Prescod for a more procedural based approach, and Michel Pelletier on a more declarative approach. Paul's group has yet to produce a PEP but has a prototype implementation, and Michel's group has a PEP posted at: http://www.zope.org/Members/michel/InterfacesPEP/PEP.txt Particular Quotes in the last 168 posts I found insightful: "The deep(er) part is whether the object passed in thinks of itself as implementing the Foo interface. This means that its author has (presumably) spent at least a little time about the invariants that a Foo should obey." -- Guido "Type errors are not even close to the majority of those I make while programming in Python... Expressiveness, IMO, is a far better aid to correctness than artificial restrictions" -- Uche "1. Better error checking (major), 2. Better documentation (major), 3. Optimization (minor), 4. Static Type Glue (minor). For now drop static type checking and higher order types. If Python's type system makes anyone's brain explode then we have not done a good job." -- Prescod on Goals "This suggestion provides a powerful (Turing-complete!) way of defining types." -- Prescod on Tim Hochberg's proposal "In short, it looks like we will need to agree on the Python type hierarchy before we can ever hope to agree on any kind of type assertions." -- Guido "We can write code quickly in Python in part because we don't have to decide in advance which methods we want to require -- and we don't have to implement every method that a builtin file has when we write a class that is a file-like object." -- Jeremy Hylton "Type Inference will never work well in Python. Python is too dynamic and the behavior of libraries wrt. types is too irregular to be inferred statically from usage (perhaps except most simple cases, which is impractical)." and -- Marcin 'Qrczak' Kowalczyk "Collections are where typechecking is most needed, and where it is least provided." -- Krishnaswami, Neel "There is no concept of asking an object which interface it implements. There is no "the" interface it implements. It's not even a set of interfaces, because the object doesn't know them in advance. Interfaces can be defined after objects conforming to them are created." -- Marcin 'Qrczak' Kowalczyk From cce@clarkevans.com Mon Mar 19 03:30:13 2001 From: cce@clarkevans.com (Clark C. Evans) Date: Sun, 18 Mar 2001 22:30:13 -0500 (EST) Subject: [Types-sig] QueryProtocol In-Reply-To: Message-ID: These two quotes are the most interesting to me: "The deep(er) part is whether the object passed in thinks of itself as implementing the Foo interface. This means that its author has (presumably) spent at least a little time about the invariants that a Foo should obey." -- Guido and "There is no concept of asking an object which interface it implements. There is no "the" interface it implements. It's not even a set of interfaces, because the object doesn't know them in advance. Interfaces can be defined after objects conforming to them are created." -- Marcin 'Qrczak' Kowalczyk Mostly beacuse I see them as complements. Indeed, this distinction between "declarative, I am a such-and-such" vs "descriptive, It looks like a such-and-such" are at the basis of Paul's proposal, with two type check methods. Further, there was question about the difference between the words "interface" and "protocol". I may be incorrect, but I'd like to align "protocol" with Guido's vision of types and "interface' with Marcin's vision of types. Thus, the only way to know if an interface is correct, is to brutally check it (not trusting the prorammer). However, one cannot check the protocol easily... thus, this falls more into the realm of intent. With these points in mind, I share Samuele Pedroni reservation about mixing these methods (and blurring protocol with interface) and I have two suggestions: (a) It may be prudent to adopt Marcin's view for an "interface type-checker" and only support the descriptive method for the PEP Prescod is preparing. The declarative check may not be sufficient (or even accurate based on Python's dynamic nature). Having both methods is perhaps confusing as Samuele asserts. (b) Introduce a new, complementary method for the declarative method. Thus, making a second PEP which is concerned about "protocol declarations" and not about "interface checking". Perhaps I'm completely wrong here, this is just my gut reaction to reading the first 168 posts on this list. ... I picture a "protocol declaration" as a dynamic discovery mechanism much like Microsoft COM QueryInterface. However, I think there is an important distinction as python does not have strict typing -- so I'm calling it QueryProtocol. For example, let's say I'm traversing a tree-like structure and I want to make some generic algorithems that do this sort of thing. I can label my protocol, "com.clarkevans.treenode" and then ask any object passed to my algorithem: node = obj.queryprotocol("com.clarkevans.treenode") And it will return to me an object (perhaps itself) which has the type of behavior that I expect. Note, unless the object *explicitly* supports the com.clarkevans.treenode protocol, this call will fail. This is declarative. ... As another example, if __queryprotocol__ was a built-in, with a protcol(object,"my.protocol") function, then the __iter__ slot of the iterator PEP could be done away with, as __iter__() would be the same as __queryprotocol__("org.python.iterator") ... Summary: Perhaps I'm wrong here, but I think there are two different kinds of "type" being tossed around here, that of a protocol (which is all about behavior and expectations) and interface (which is about function signatures). Protocols are best supported via declaration and discovery, while Interfaces are best checked via descriptive techniques. The declarative, protcol method can be implemented through a simple "query" interface. Clark P.S. QueryProtocol is _not_ the same as coersion! From paulp@ActiveState.com Mon Mar 19 07:46:19 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Sun, 18 Mar 2001 23:46:19 -0800 Subject: [Types-sig] Type annotation progress Message-ID: <3AB5B94A.370A4752@ActiveState.com> I've now got a proper grammar for type declarations. I also have the ability to allow out-of-line declarations so that I can annotate the standard library without actually editing the files. For your interest, here is a declaration file for "sre" (also known as "re"). I have a script that compiles these declarations into the .pyc associated with a module. SRE passes all test cases after this process and yet gives proper error checks when misused: python -c "import sre; print sre.match(None, 'foo') Traceback (most recent call last): File "", line 1, in ? File "c:\python20\lib\sre.py", line 43, in match def match(pattern, string, flags=0): typecheck.InterfaceError: Parameter 'pattern' expected Unicode or 8-bit string. Instead it got 'None' (None) ========================== __declare__(""" def compile(pattern: IString, flags: IInt) -> compiled_re: IRegex def match(pattern: IString, string: IString, flags: IInt) -> match : (IMatch | INone) def search(pattern: IString, string: IString, flags: IInt) -> match : (IMatch | INone) def split(pattern: IString, string: IString, maxsplit: IInt) -> ISequence def findall(pattern: IString, string: IString) -> ISequence def sub(pattern: IString, repl: (IString|ICallable), string: IString, count: IInt) -> IString def subn(pattern: IString, repl: IString, count: IInt) -> (new_string: IString, number_of_subs_made: IInt) def escape(pattern: IString) -> escaped_string: IString """) -- Take a recipe. Leave a recipe. Python Cookbook! http://www.activestate.com/pythoncookbook From cce@clarkevans.com Mon Mar 19 08:17:34 2001 From: cce@clarkevans.com (Clark C. Evans) Date: Mon, 19 Mar 2001 03:17:34 -0500 (EST) Subject: [Types-sig] Indirect, Explicit Interface In-Reply-To: Message-ID: I was thinking of a way to specify and use Interfaces that was not expressed recently... (but may have been in the previous two year history). interface com.clarkevans.monty.eggs: def eggs(string): "Takes a string and prints out eggs" interface com.clarkevans.monty.ham: def spam(com.clarkevans.monty.eggs, string): "Takes an object implementing eggs, and a string giving ham" class EggsOnly: def eggs(str): print "eggs!" + str def __queryinterface__(self,anInterface): if anInterface == "com.clarkevans.monty.eggs": return self raise "Unsupported Interface" class EggsAndSpam: def spam(eggs,str): eggs.eggs(str) print " and ham" def __queryinterface__(self,anInterface): if anInterface == "com.clarkevans.monty.eggs": return EggsOnly() if anInterface == "com.clarkevans.monty.ham": return self This may not be the best exposition but here are the salient points: 1. A new type keyword is added (interface) 2. def within an interface is used to define methods only 3. within the interface def, arguments are actual interface names 4. built-in names lack periods, "string", names not built-in are in some unique syntax, like Java Package style, for instance. 5. No static type marker, instead there is a dynamic queryinterface In any case, as a newbie user, I think I like arguments not having type markers like C and Java. Type markers clutter everything up. Also, variable names are often useless in an interface... so I was thinking type names would be better here. Hope this helps! Clark From paulp@ActiveState.com Mon Mar 19 08:38:07 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Mon, 19 Mar 2001 00:38:07 -0800 Subject: [Types-sig] Indirect, Explicit Interface References: Message-ID: <3AB5C56F.84A417F9@ActiveState.com> "Clark C. Evans" wrote: > > ... > > In any case, as a newbie user, I think I like arguments > not having type markers like C and Java. Type markers > clutter everything up. Also, variable names are often > useless in an interface... so I was thinking type names > would be better here. My feeling is exactly opposite. Parameter *names* are more important for figuring out how to use a parameter because given the name you can often figure out the type but the reverse is not true. Plus, Python has a keyword arguments feature that allows values to change their order. Also, names are what you refer to from the documentation. In my declaration grammar, types are optional but names are required. The "default type" is "Anything." -- Take a recipe. Leave a recipe. Python Cookbook! http://www.activestate.com/pythoncookbook From paulp@ActiveState.com Mon Mar 19 09:22:21 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Mon, 19 Mar 2001 01:22:21 -0800 Subject: [Types-sig] Interface PEP References: <200103151449.PAA00252@core.inf.ethz.ch> Message-ID: <3AB5CFCD.923EE1E9@ActiveState.com> [feel free to keep insisting. you are helping to sharpen my thinking on these issues] Samuele Pedroni wrote: > > Hi. > >... > > All interfaces are domain but at least at the beginning of our > discussion not all domains were interfaces, one could define > the domain Prime. I agree with this. The procedural definition is always more general than the declarative ones because you have the full power of Python available for your declaration. > So I don't know if the interface syntax can > be stretched to the point to deal with that too and Prime > is more an algorithmic-checkable property than something that an > input should declare to implement. Fair enough. > OTOH yes as far as protocol are concerned one > could use the interface syntax to define them. > So we can the following: > > interface Readable: > def read(size) > > than there is an ambiguity problem when we should enforce the check: > with your new distinction __check__ vs. __implementedby__ should: > > def grab(r: Readable) > ... > > test Readable.__check__(r) or Readable.__implementedby__(r). > Perhaps there needs to be a "strict" modifier to declarations that says whether merely checking for methods is sufficient or whether an explicit declaration is required. It should probably be up to a specific function whether it wants to use the first method of checking, or the second, or both. On the other hand, perhaps it is just extra complexity. If you wanted strict checking, wouldn't it be easy to implement an __check__ method that checked both the __interfaces__ AND the actual signature? Or you could require a method name like def __supports_readability__(): pass. Python already has types and classes. I'm adding something that you call protocols. Michel is adding something that he calls interfaces. It makes sense to have type declarations that reference all four. It is important to me to simplify this model... -- Take a recipe. Leave a recipe. Python Cookbook! http://www.activestate.com/pythoncookbook From paulp@ActiveState.com Mon Mar 19 09:24:04 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Mon, 19 Mar 2001 01:24:04 -0800 Subject: [Types-sig] QueryProtocol References: Message-ID: <3AB5D034.6295A610@ActiveState.com> QueryProtocol is an idea deserving of a PEP. I think it is interesting but I would like to see it commonly used as a convention in Python programming before it became a part of the language. -- Take a recipe. Leave a recipe. Python Cookbook! http://www.activestate.com/pythoncookbook From paulp@ActiveState.com Mon Mar 19 09:36:27 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Mon, 19 Mar 2001 01:36:27 -0800 Subject: [Types-sig] "Concrete types" References: <3AAFCBDF.39A0735D@home.se> <3AB00D5A.453E7BCF@home.se> Message-ID: <3AB5D31B.AD0989@ActiveState.com> I think that there is an important point hiding in the middle of a long message. :) Marcin 'Qrczak' Kowalczyk wrote: > > Because an interface rarely says anything about the representation. > Other types supporting the same interface should be generally accepted > where lists are accepted. ListType says too much about an object. > Python has exceptions to this. Many builtin types require other builtin > types, e.g. __dict__ of a builtin class object must have the builtin > dictionary type. It's slowly shifting to allow more generality, but... > > ...concrete types must be finally used somewhere. You can express > an abstract interface of an integer, but you can't express every > abstract interface in terms of other abstract interfaces. Finally > some real work must be done besides message passing. > If you try to express pure Smalltalk model in Haskell: > > newtype Object = O (String -> [Object] -> IO Object) > > then you can't do anything useful unless you encode data in method > names, because the only thing you can do with an object is to send > it a message, so you can't know which damn integer number an object > represents to implement arithmetic or to index a sequence. Some > objects or some messages must be special. > > There is a balance between concrete and abstractness. It's impossible > to be completely abstract, and should not be completely concrete > because it's very painful to use (as in C and Pascal). This is an excellent point. We should not go overboard and require that everywhere an integer can be used an integer-like object should also be usable. This is not reasonable for objects that are only ever worked *on* and have no intrinsic behavior or their own. -- Take a recipe. Leave a recipe. Python Cookbook! http://www.activestate.com/pythoncookbook From Samuele Pedroni Mon Mar 19 14:05:39 2001 From: Samuele Pedroni (Samuele Pedroni) Date: Mon, 19 Mar 2001 15:05:39 +0100 (MET) Subject: [Types-sig] Interface PEP Message-ID: <200103191405.PAA04441@core.inf.ethz.ch> Hi. > > Perhaps there needs to be a "strict" modifier to declarations that says > whether merely checking for methods is sufficient or whether an explicit > declaration is required. It should probably be up to a specific function > whether it wants to use the first method of checking, or the second, or > both. > > On the other hand, perhaps it is just extra complexity. If you wanted > strict checking, wouldn't it be easy to implement an __check__ method > that checked both the __interfaces__ AND the actual signature? Or you > could require a method name like def __supports_readability__(): pass. > > Python already has types and classes. I'm adding something that you call > protocols. Michel is adding something that he calls interfaces. It makes > sense to have type declarations that reference all four. It is important > to me to simplify this model... > Yes, this is overcomplicated to fit in python. But understanding that there are in principle differences is important. [This is mostly a repetion but ...] - Your idea idea for unifying protocols and interfaces is to check both for isImplementedBy first and for signatures. My opinion was that checking for interfaces should simply check isImplementedBy, why: - because it makes sense for the interface philosophy (which OTOH has little to do with python plain old way-of-life). - because checking just the signature in the case of tiny interfaces with few methods having inexpressive names make typechecking weak when that is not required. And this kind of interfaces are used and written (at least in java which is mainstream and have interfaces that resemble the proposed one). - we can guarantee signature comformance at class def time. Protocols: mostly needed to typecheck the std lib, and is worth to notice that things like Mapping or Sequence (because of type/class dicotomy) cannot be expressed really as Michel Pelletier interfaces. They will be mostly predefined or be very tiny, I see little chance that someone can positively define new complex protocols outside the std lib tradition => we don't really need a nice syntax to define them, an anonymous constructor can do the job, or a function that take a class/interface and return the corresponding protocol. IMHO the point is that there is no need to weaken interface typechecking (if we will have them) for the sake of protocols. Hoping in the type/class unification we can live just with class,interfaces and domains (something offering a __check__ for typechecking; if we agree that checking for something like Prime makes sense), then we should simply offer a way to construct a domain that check for signatures (a "protocol" if we want). regards. From Samuele Pedroni Mon Mar 19 14:24:59 2001 From: Samuele Pedroni (Samuele Pedroni) Date: Mon, 19 Mar 2001 15:24:59 +0100 (MET) Subject: [Types-sig] Interface PEP Message-ID: <200103191424.PAA05001@core.inf.ethz.ch> Some other motivations. Interfaces are declarive things (aliens coming from other worlds, if you want), proposed to support complicated designs using reflection: not really newbies stuff, in their original worlds they are checked against explicit declarations. I think it is better to keep their philosophy for them. Everything else is python runtime, algorithmic and dynamic style. I wonder if mixing the two just simplify implementation or really help understanding. My idea is that if for specifying the protocols kind of typechecking against signatures one should write something algorithmic costs and approach will be more clear. regards. From paulp@ActiveState.com Mon Mar 19 14:35:17 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Mon, 19 Mar 2001 06:35:17 -0800 Subject: [Types-sig] Another declaration file Message-ID: <3AB61925.4999E0A5@ActiveState.com> This one is for xmllib. It's funny how doing this shows the extent to which people write code with the expectation that parameter names are private (even though they aren't in Python). For instance xmllib.XMLParser uses keyword arguments instead of bothing to enumerate its arguments. And it has parameters named "i" and "j". =================================================== from typecheck import __declare__ from sre import IMatch __declare__(""" # xmllib has a funny way of doing its init... #def XMLParser.__init__(self, # accept_unquoted_attributes: IStrictBoolean, # accept_missing_endtag_name: IStrictBoolean, # map_case: IStrictBoolean, # accept_utf8: IString, # translate_attribute_references: IStrictBoolean ) def XMLParser.reset(self) -> rc: INone def XMLParser.setnomoretags(self) -> rc: INone def XMLParser.setliteral(self) -> rc: INone def XMLParser.feed(self, data: IString) -> rc: INone def XMLParser.close(self) -> rc: INone def XMLParser.translate_references(self, data:IString, all: IStrictBoolean) -> unknown: (IString | INone) def XMLParser.getnamespace(self) -> namespace: IMapping def XMLParser.goahead(self, end: IStrictBoolean) -> rc: INone def XMLParser.parse_comment(self, i: IInt) -> length_or_no_end : IInt def XMLParser.parse_doctype(self, res: IMatch) -> length_or_no_end : IInt def XMLParser.parse_cdata(self, i: IInt) -> length_or_no_end : IInt def XMLparser.parse_starttag(self, i: IInt) -> length_or_no_end : IInt def XMLParser.parse_endtag(self, i: IInt) -> length_or_no_end : IInt def XMLParser.finish_starttag(self, tagname: IString, attrdict: IMapping, method: (ICallable | INone) ) -> length_or_no_end : IInt def XMLParser.finish_endtag(self, tag: IString) -> rc: INone def XMLParser.handle_charref(self, name: IString) -> rc: INone def XMLparser.parse_proc(self, i: IInt) -> length_or_no_end : IInt def XMLParser.parse_attributes(self, tag: IString, i: IInt, j: IInt) -> (attrdict: IMapping, namespace: IMapping, i: IInt) """) -- Take a recipe. Leave a recipe. Python Cookbook! http://www.activestate.com/pythoncookbook From paulp@ActiveState.com Mon Mar 19 14:38:58 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Mon, 19 Mar 2001 06:38:58 -0800 Subject: [Types-sig] Interface PEP References: <200103191405.PAA04441@core.inf.ethz.ch> Message-ID: <3AB61A02.A3EECA4A@ActiveState.com> Samuele Pedroni wrote: > > ... > > My opinion was that checking for interfaces should simply check isImplementedBy, > why: > - because it makes sense for the interface philosophy (which OTOH > has little to do with python plain old way-of-life). Okay, we can try that and see if anyone screams. I am happy to start out strict and then relax later. -- Take a recipe. Leave a recipe. Python Cookbook! http://www.activestate.com/pythoncookbook From Samuele Pedroni Mon Mar 19 14:43:02 2001 From: Samuele Pedroni (Samuele Pedroni) Date: Mon, 19 Mar 2001 15:43:02 +0100 (MET) Subject: [Types-sig] Interface PEP Message-ID: <200103191443.PAA05409@core.inf.ethz.ch> Hi. > Samuele Pedroni wrote: > > > > ... > > > > My opinion was that checking for interfaces should simply check isImplementedBy, > > why: > > - because it makes sense for the interface philosophy (which OTOH > > has little to do with python plain old way-of-life). > > Okay, we can try that and see if anyone screams. I am happy to start out > strict and then relax later. Clearly with interfaces I was referring to Michel Pelletier interfaces, y with interfaces I was referring to Michel Pelletier interfaces, not to Mapping, Sequences, file-like obj expected by the lib which them we *must* be loose ;). From Samuele Pedroni Mon Mar 19 14:59:23 2001 From: Samuele Pedroni (Samuele Pedroni) Date: Mon, 19 Mar 2001 15:59:23 +0100 (MET) Subject: [Types-sig] Interface PEP Message-ID: <200103191459.PAA05768@core.inf.ethz.ch> > Hi. > > > Samuele Pedroni wrote: > > > > > > ... > > > > > > My opinion was that checking for interfaces should simply check > isImplementedBy, > > > why: > > > - because it makes sense for the interface philosophy (which OTOH > > > has little to do with python plain old way-of-life). > > > > Okay, we can try that and see if anyone screams. I am happy to start out > > strict and then relax later. > > Clearly with interfaces I was referring to Michel Pelletier interfaces, > y with interfaces I was referring to Michel Pelletier interfaces, > not to Mapping, Sequences, file-like obj expected by the lib > which them we *must* be loose ;). A readable version: Clearly with interfaces I was referring to Michel Pelletier interfaces, not to Mapping, Sequences, file-like obj expected by the std lib with them we *must* be loose ;). So for the moment given that we do not have an implementation for Michel Pelletier interfaces nobody should scream. On the other hand I see that we have a problem with C Extensions and things they produce: how do you hav defined IMatch? I imagine it should deal with the results of both sre and old re and isImplementedBy cannot do the job ;). regards. From paulp@ActiveState.com Mon Mar 19 15:23:25 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Mon, 19 Mar 2001 07:23:25 -0800 Subject: [Types-sig] Interface PEP References: <200103191459.PAA05768@core.inf.ethz.ch> Message-ID: <3AB6246D.6B534E07@ActiveState.com> Samuele Pedroni wrote: > >... > > On the other hand I see that we have a problem with C Extensions and > things they produce: how do you hav defined IMatch? I imagine > it should deal with the results of both sre and old re and isImplementedBy > cannot do the job ;). # should do better interfaces for these IMatch = Interface("IMatch", "Regular expression match", lambda obj: hasattr(obj, "group")) IRegex = Interface("IRegex", "Regular expression object", lambda obj: hasattr(obj, "match")) But note that these are of rather marginal interest. They are most often the *return value* rather than the input of a function. The simple checks I do above are sufficient to ensure that where they ARE used as paramaters, you will not accidently pass one for the other or a string for either. So the most likely mistakes are removed. Fulton/Pelletier interfaces could make these objects more introspectable but the simple definitions are sufficient for finding simple mistakes. Also, it isn't quite clear to what degree they are really polymophically replacable. I'd have to do a survey of places where they are used as inputs to code to determine whether I should just use the concrete types (i.e. perhaps "sre" and "pre" matches are considered incompatible) or have an abstraction. Really, domain experts should be design the type signatures for each module! -- Take a recipe. Leave a recipe. Python Cookbook! http://www.activestate.com/pythoncookbook From qrczak@knm.org.pl Mon Mar 19 18:10:37 2001 From: qrczak@knm.org.pl (Marcin 'Qrczak' Kowalczyk) Date: 19 Mar 2001 18:10:37 GMT Subject: [Types-sig] Interface PEP References: <200103151449.PAA00252@core.inf.ethz.ch> <3AB5CFCD.923EE1E9@ActiveState.com> Message-ID: Mon, 19 Mar 2001 01:22:21 -0800, Paul Prescod pis= ze: > Perhaps there needs to be a "strict" modifier to declarations that > says whether merely checking for methods is sufficient or whether > an explicit declaration is required. It should probably be up to > a specific function whether it wants to use the first method of > checking, or the second, or both. >=20 > On the other hand, perhaps it is just extra complexity. IMHO it's unnecessary complexity. A good language should not support a wide variety of flavors of a feature, but one well defined flavor which can cover the variety inside its framework. In this case there should be one flavor of "checking whether the object conforms to an interface". Interfaces themselves can be more or less strict, and there can even be a way to transform in one direction or another if it makes sense in general. > Python already has types and classes. I'm adding something that > you call protocols. Michel is adding something that he calls > interfaces. It makes sense to have type declarations that reference > all four. It is important to me to simplify this model... Indeed. So instead of saying - how to check a type - how to check a class - how to check a protocol - how to check an interface we should say - how to check an interface (or whatever it's called) - how to make an interface from a type - how to make an interface from a class (should be similar to the type) - how to make an interface from a protocol (or whatever) (sorry, I don't remember which are protocols and which are interfaces). In other words, the interface of an interface should be simple. Interfaces themselves can be complex and built in sophisticated ways. --=20 __("< Marcin Kowalczyk * qrczak@knm.org.pl http://qrczak.ids.net.pl/ \__/ ^^ SYGNATURA ZAST=CAPCZA QRCZAK From tony@adam12.metanet.com Tue Mar 20 16:27:12 2001 From: tony@adam12.metanet.com (Tony Lownds) Date: Tue, 20 Mar 2001 08:27:12 -0800 Subject: [Types-sig] Another declaration file In-Reply-To: <3AB61925.4999E0A5@ActiveState.com> References: <3AB61925.4999E0A5@ActiveState.com> Message-ID: How do these declarations become real run-time checks? Does it appear in the XMLParser.py file or a separate file? Do you still need to add __paramcheck__() and __rccheck__() calls to the functions? Can you post your latest code? -Tony >=================================================== >from typecheck import __declare__ >from sre import IMatch > >__declare__(""" > ># xmllib has a funny way of doing its init... >#def XMLParser.__init__(self, ># accept_unquoted_attributes: IStrictBoolean, ># accept_missing_endtag_name: IStrictBoolean, ># map_case: IStrictBoolean, ># accept_utf8: IString, ># translate_attribute_references: IStrictBoolean ) > >def XMLParser.reset(self) -> rc: INone > >def XMLParser.setnomoretags(self) -> rc: INone > >def XMLParser.setliteral(self) -> rc: INone > >def XMLParser.feed(self, data: IString) -> rc: INone > >def XMLParser.close(self) -> rc: INone > >def XMLParser.translate_references(self, > data:IString, all: IStrictBoolean) > -> unknown: (IString | INone) > >def XMLParser.getnamespace(self) -> namespace: IMapping > >def XMLParser.goahead(self, end: IStrictBoolean) -> rc: INone > >def XMLParser.parse_comment(self, i: IInt) -> length_or_no_end : IInt > >def XMLParser.parse_doctype(self, res: IMatch) > -> length_or_no_end : IInt > >def XMLParser.parse_cdata(self, i: IInt) -> length_or_no_end : IInt > >def XMLparser.parse_starttag(self, i: IInt) > -> length_or_no_end : IInt > >def XMLParser.parse_endtag(self, i: IInt) -> length_or_no_end : IInt > >def XMLParser.finish_starttag(self, tagname: IString, > attrdict: IMapping, method: (ICallable | INone) ) > -> length_or_no_end : IInt > >def XMLParser.finish_endtag(self, tag: IString) -> rc: INone > >def XMLParser.handle_charref(self, name: IString) -> rc: INone > > >def XMLparser.parse_proc(self, i: IInt) -> length_or_no_end : IInt > >def XMLParser.parse_attributes(self, tag: IString, > i: IInt, j: IInt) > -> (attrdict: IMapping, namespace: IMapping, i: IInt) > >""") >-- >Take a recipe. Leave a recipe. >Python Cookbook! http://www.activestate.com/pythoncookbook > >_______________________________________________ >Types-SIG mailing list >Types-SIG@python.org >http://mail.python.org/mailman/listinfo/types-sig From tim.hochberg@ieee.org Tue Mar 20 17:59:05 2001 From: tim.hochberg@ieee.org (Tim Hochberg) Date: Tue, 20 Mar 2001 10:59:05 -0700 Subject: [Types-sig] Some more type annotations stuff References: <3AB5B94A.370A4752@ActiveState.com> Message-ID: <03f401c0b167$74a1c300$87740918@CX781526B> This is a multi-part message in MIME format. ------=_NextPart_000_03F1_01C0B12C.C811EFF0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Sorry I've been missing from the discussion. The harddrive on my laptop is doing a slow motion meltdown, so I'm distracted. Attached is my latest try at putting together check methods for immuttable sequences and mappings. I also include preliminary support for anonymous typechecking (I_getitem, I_setitem, etc). For mutable sequences and mappings things get much murkier. I took a stab at checking for __setitem__ (see I_setitem), but as you can see if you look at the code, the method is not entirely satisfactory. It sounds like Paul is making good progress on the typechecking code, so things are looking up. -tim ------=_NextPart_000_03F1_01C0B12C.C811EFF0 Content-Type: text/plain; name="typecheckstuff.py" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="typecheckstuff.py" # March 14: Immutable Sequences # March 15: Fixed(?) iterable protocol to forbid mappings without iter = protocol # March 16: Immutable mappings, fixes several bugs. testing. =20 import types, operator, typecheck, copy =20 class HorribleTypeError(Exception): pass def _getitem(object): """Checks if an object supports __getitem__. Assumes that item state is not changed by getitem. """ try: object[0] return 1 except (IndexError, KeyError): return 1 except Exception, err: import traceback traceback.print_exc() return 0 I_getitem =3D typecheck.Interface("I_getitem", "Any object that supports = getitem", _getitem) def _len(object): """Checks if an object supports __len__""" try: len(object) return 1 except: return 0 =20 I_len =3D typecheck.Interface("I_len", "Any object that supports len", = _len) def _setitem(object): """Checks if an objects supports __setitem This is potentially dangersous. We make an effort to avoid modifying object, but it could happen. """ try: object[0] =3D object[0] return 1 except IndexError: # This is a sequence-like object try: # This should fail with an IndexError if it's setable object[0] =3D None except IndexError: return 1 except: return 0 except KeyError: # This is a mapping-like object # The only way I can think of to more-or-less nondestructively = test requires delitem # Try and hope for the best. try: object[0] =3D None del object[0] return 1 except: raise HorribleTypeError("Nondestructive setitem test failed = object may be corrupted!") except: return 0 I_setitem =3D typecheck.Interface("I_setitem", "Any object that supports = setitem", _setitem) def _iterable(object): """An iterable implements __getitem__ or the iteration protocol = (TBD) """ # if implements_iter_protocol(object): return 1 # # Exclude mappings that don't implement iteration if hasattr(object, "keys"): return 0 # Anything else should be a sequence and thus iteratable. if hasattr(object, "__getitem__"): return 1 # This isn't really right, what we want to do is to check whether # object[0] will work. However using the try: object[0] approach # man not work correctly. If the object uses the # current iteration protocol calling object[0] may # advance the iteration.=20 if not type(object) =3D=3D types.InstanceType and \ operator.isSequenceType(object): return 1 return 0 IIterable =3D typecheck.Interface("IIterable", _iterable.__doc__, = _iterable) def _basicSequence(object): """A basic sequence implements __getitem__ and __len__ and does not = implement keys. This last condition is necessary to distinguish a sequence from a = mapping. This is sufficient to build a tuple, but doesn't necessarily support concatenation, repitition or slicing. """ # Even though I refused to use the basic getitem check for the # iterable protocol, it's probably OK here. Objects with a length # attribute should not be changed by accessing an item. return _len(object) and _getitem(object) and not hasattr(object, = "keys") IBasicSequence =3D typecheck.Interface("IBasicSequence", = _basicSequence.__doc__, _basicSequence) def _sequence(object): """A full sequence implements the same methods as a tuple. A sequence can always be constructed from a BasicSequence, see SequenceWrapper below. """ if not _basicSequence(object): return 0 try: # Check for repitition and concatenation. # I'm hoping this is less expensive for long sequences. 0*object + 0*object object[:0] except: return 0 return 1 ISequence =3D typecheck.Interface("ISequence", _sequence.__doc__, = _sequence) def _basicMapping(object): """A basic mapping supports getitem and keys""" return _getitem(object) and hasattr(object, "keys") IBasicMapping =3D typecheck.Interface("IBasicMapping", = _basicMapping.__doc__, _basicMapping) def _mapping(object): """A mapping supports getitem, keys, len, copy, has_key, items, = values and get . These are all the items a dictionary would support if it were = immutable. """ return (_basicMapping(object) and _len(object) and hasattr(object, "copy") and hasattr(object, "has_key") and hasattr(object, "items") and hasattr(object, "values") and hasattr(object, "get")) IMapping =3D typecheck.Interface("IMapping", _mapping.__doc__, _mapping) # It's kind of obnoxious having both BasicSequence and SequenceBase. One = or both should # probably be renamed. class SequenceBase: """Class to simplify derivation of objects that satisfy the full = sequence protocol. This class provides the same functionality as a tuple. At a minimum, __getitem__ and __len__ must be overridden. = Concatenation and repition return tuples by default. If other behaviour is desired, override = __add__ and __mul__ as well. Subclasses may also want to override __getslice__ for efficiency. =20 """ # The following list is based on section 2.1.5 in the library = reference # x in s: Should be take care of by the sequence prototcol(?) # x not in s: Ditto # s + t: def __add__(self, other): return tuple(self) + tuple(other) # s * n, n * s: def __mul__(self, n): return tuple(self)*n __rmul__ =3D __mul__ # s[i] def __getitem__(self, i): raise NotImplementedError # s[i:j] def __getslice__(self, start, stop): l =3D [] for i in range(start, stop): l.append(self[i]) return tuple(l) # len(s) def __len__(self): raise NotImplementedError # min(s) # max(s) These two work automagically for any iterable. class SequenceWrapper(SequenceBase): def __init__(self, wrapped): self.wrapped =3D wrapped def __getitem__(self, i): return self.wrapped[i] def __len__(self): return len(self.wrapped) class MappingBase: """Class to simplify the derivation of classes that satisfy the = Mapping protocol. This class supplies all the methods of a standard disctionary that do not require mutation. The user must overided __getitem__ and keys. Other methods may be overridden for efficiency. =20 """ def __getitem__(self, key): raise NotImplementedError def keys(self): raise NotImplementedError def __len__(self): return len(self.keys()) def copy(self): pass def has_key(self,key): try: self[key] return 1 except KeyError: return 0 def items(self): return [(key, self[key]) for key in self.keys()] def values(self): return [self[key] for key in self.keys()] def get(self, key, default=3DNone): if self.has_key(key): return self[key] else: return default class MappingWrapper(MappingBase): def __init__(self, wrapped): self.wrapped =3D wrapped def __getitem__(self, i): return self.wrapped[i] def keys(self): return self.wrapped.keys() # Very basic testing. class _aniterable: def __getitem__(self, i): if 0 <=3D i < 5: return i raise IndexError anInterable =3D _aniterable() class _aBasicSequence(_aniterable): def __len__(self): return 5 aBasicSequence =3D _aBasicSequence() aSequence =3D SequenceWrapper(aBasicSequence) class _aBasicMapping(_aniterable): def keys(self): return range(5) aBasicMapping =3D _aBasicMapping() aMapping =3D MappingWrapper(aBasicMapping) for obj, isIter, isBasic, isSeq, hasGI, hasLen, hasSI, isBMap, isMap in = [ ((),1, 1, 1, 1, 1, 0, 0, 0), ([],1, 1, 1, 1, 1, 1, 0, 0), (anInterable, 1, 0, 0, 1, 0, 0, 0, = 0), (aBasicSequence, 1, 1, 0, 1, 1, 0, = 0, 0), (aSequence, 1, 1, 1, 1, 1, 0, 0, = 0), (aBasicMapping, 0, 0, 0, 1, 0, 0, = 1, 0), (aMapping, 0, 0, 0, 1, 1, 0, 1, 1), = =20 ({}, 0, 0, 0, 1, 1, 1, 1, 1)]: orig =3D copy.deepcopy(obj) assert IIterable.__check__(obj) =3D=3D isIter, "%s failed (expected = %s" % (obj,isIter) assert IBasicSequence.__check__(obj) =3D=3D isBasic, "%s failed = (expected %s)" % (obj,isBasic) assert ISequence.__check__(obj) =3D=3D isSeq, "%s failed" % (obj,) assert I_getitem.__check__(obj) =3D=3D hasGI, "%s failed" % (obj,)=20 assert I_len.__check__(obj) =3D=3D hasLen, "%s failed (expected %s)" = % (obj,hasLen)=20 assert I_setitem.__check__(obj) =3D=3D hasSI, "%s failed" % (obj,) assert IBasicMapping.__check__(obj) =3D=3D isBMap, "%s failed = (expected %s)" % (obj,isBMap) =20 assert IMapping.__check__(obj) =3D=3D isMap, "%s failed" % (obj,) = if _iterable(obj): for a, b in zip(obj, orig): assert a =3D=3D b, "%s changed (%s !=3D %s)" % (obj, a, b) elif type(obj) =3D=3D type({}): assert obj =3D=3D orig, "%s !=3D %s" % (obj, orig) print obj, "(", isIter, isBasic, isSeq, hasGI, ") passed" ------=_NextPart_000_03F1_01C0B12C.C811EFF0-- From cce@clarkevans.com Tue Mar 20 23:56:11 2001 From: cce@clarkevans.com (Clark C. Evans) Date: Tue, 20 Mar 2001 18:56:11 -0500 (EST) Subject: [Types-sig] QueryProtocol In-Reply-To: <3AB5D034.6295A610@ActiveState.com> Message-ID: On Mon, 19 Mar 2001, Paul Prescod wrote: > QueryProtocol is an idea deserving of a PEP. I think it is interesting > but I would like to see it commonly used as a convention in Python > programming before it became a part of the language. Ok. I hope this is adequate. ;) Clark ---------- Forwarded message ---------- Date: Tue, 20 Mar 2001 18:43:48 -0500 (EST) From: Clark C. Evans To: python-list@python.org Newsgroups: comp.lang.python Subject: Yet Another PEP: Query Protocol Interface or __query__ Please have a look at the PEP below, it is intended to be a light-weight solution to "typing" based on programmer intent. Regards, Clark Evans PEP: XXX Title: Query Interface Protocol Version: $Revision$ Author: Clark Evans Python-Version: 2.2 Status: Active Type: Standards Track Created: 21-Mar-2001 Summary This paper asserts that "interface typing" can be carved into two separable concerns, that of (a) protocol, which is all about behavior/expectations, and that of (b) signature which is about method existence and argument type checking. This proposal puts forth a declarative method for interface protocol discovery that could be orthogonal and complementary to a more signature based approach as being developed by the types special interest group. Context Python is a very dynamic language with powerful introspection capabilities. However, it has yet to formalize an interface or abstract type checking mechanism. A consensus for a user defined type system may be far off into the future. Currently, existence of particular methods, particularly those that are built-in such as __getitem__, is used as an indicator of support for a particular interface. This method may work for interfaces blessed by GvR, such as the new enumerator interface being proposed and identified by a new built-in __iter__. However, this current method does not admit an infallible way to identify interfaces lacking a built-in method. Motivation In the recent type special interest group discussion [1], there were two complementary quotes which motivated this proposal: "The deep(er) part is whether the object passed in thinks of itself as implementing the Foo interface. This means that its author has (presumably) spent at least a little time about the invariants that a Foo should obey." - GvR [2] and "There is no concept of asking an object which interface it implements. There is no "the" interface it implements. It's not even a set of interfaces, because the object doesn't know them in advance. Interfaces can be defined after objects conforming to them are created." -- Marcin Kowalczyk [3] The first quote focuses on the intent of a class, including not only the existence of particular methods, but more importantly the call sequence, behavior, and other invariants. Where the second quote focuses on the type signature of the class. These quotes motivate a distinction between interface as a "declarative, I am a such-and-such" construct, as opposed to a "descriptive, It looks like a such-and-such" mechanism. Furthermore, it is clear that both aspects of interface are important, if not completely orthogonal and complementary. For purposes of this proposal, the word "protocol" is aligned with Guido's deep issue, and "signature" with Marcin's vision of a type system. In this way, we clearly demarcate the two attitudes of the word "interface" in a type system. Clearly, detailing an exhaustive method of interface signatures is very difficult problem, especially for a dynamic language like Python where the interface signature for a class may change over time. However, a simple declarative mechanism to inquire which interface protocols an object supports at a given time is relatively straight forward and could bring immediate benefit. Details A new built-in method, __query__ is proposed. This method has a single argument, an interface protocol identifier, and either returns an object supporting the given protocol, or throws an unknown exception. An interface protocol identifier is a lower-case string having a reverse DNS host name string: TLD '.' DOMAIN ['.' NAME ]* Where TLD is a top level domain such as 'com' or 'uk.co'. DOMAIN is a registered name in the given TLD. And one or more optional NAME separated by a period. NAME is a sequence of one or more alphabetic characters including the dash '-' character. See the relevant ITEF specifications for specific details. Note that all of the protocols above have at least one period. In the future it may be prudent to introduce one or more "blessed" interface protocol identifiers which do not have a period. Potential examples include "enumerator", "sequence", "map", "string", etc. In addition, a built-in query method could be introduced that calls the __query__ method on a given object. Example Usage >>> class EggsOnly: def eggs(self,str): print "eggs!" + str def __query__(self,protocol): if protocol == "org.python.example.eggs": return self raise "unknown: " + protocol >>> class EggsAndHam: def ham(self,str): print "ham!"+str def __query__(self,protocol): if protocol == "org.python.example.ham": return self if protocol == "org.python.example.eggs": return EggsOnly() raise "unknown" + protocol >>> x = EggsOnly() >>> x.eggs("hello") eggs!hello >>> x.__query__("org.python.example.eggs").eggs("hello") eggs!hello >>> z = EggsAndHam() >>> z.ham("hi") ham!hi >>> z.__query__("org.python.example.ham").ham("hi") ham!hi >>> z.__query__("org.python.example.eggs").eggs("hi") eggs!hi >>> y = x.__query__("org.python.bad") Traceback (innermost last): File "", line 1, in ? File "", line 5, in __query__ unknown: org.python.bad Relationships: The iterator special interest group is proposing a new built-in called "__iter__", which could be replaced with __query__ and a blessed interface protocol identifier of "enumerator". Therefore calls to obj.__iter__() could be replaced with obj.__query__("enumerator") with no semantic difference. Although this proposal may sounds similar to Microsoft's QueryInterface, it differs by a number of aspects. First, there is not a special "IUnknown" interface which can be used for object identity, although this could be proposed as one of those "special" blessed interface protocol identifiers. Second, with QueryInterface, once an object supports a particular interface it must always there after support this interface; this proposal makes no such guarantee, although this may be added at a later time. Third, implementations of Microsoft's QueryInterface must support a kind of equivalence relation. By reflexive they mean the querying an interface for itself must always succeed. By symmetrical they mean that if one can successfully query an interface IA for a second interface IB, then one must also be able to successfully query the interface IB for IA. And finally, by transitive they mean if one can successfully query IA for IB and one can successfully query IB for IC, then one must be able to successfully query IA for IC. Ability to support this type of equivalence relation should be encouraged, but may not be possible. Further research on this topic (by someone familiar with Microsoft COM) would be helpful in further determining how compatible this proposal is. Backwards Compatibility There should be no problem with backwards compatibility. Indeed this proposal, save an built-in query() function, could be tested without changes to the interpreter. Future Compatibility It appears that this proposal could be implemented orthogonal to the protocol checking system being constructed [4] by Paul Prescod and company. It may not be all that compatible with the more ambitious signature declaration and checking approach[5] taken by Michel Pelletier. This requires further investigation. Questions and Answers Q: This is just a type-coercion proposal. A: No. Certainly it could be used for type-coercion, such coercion would be explicit via __query__ or query function. Of course, if this was used for iterator interface, then the for construct may do an implicit __query__("enumerator") but this would be an exception rather than the rule. Q: Why did the author write this PEP? A: He wanted a simple proposal that covered the "deep part" of interfaces without getting tied up in signature woes. Also, it was clear that __iter__ proposal put forth is just an example of this type of interface. Further, the author is doing XML based client server work, and wants to write generic tree based algorithms that work on particular interfaces and would like these algorithms to be used by anyone willing to make an "adapter" having the interface required by the algorithm. Q: Why not call this __queryinterface__ ? A: Too close to Microsoft's QueryInterface, especially given the semantic differences which may not be reconcilable. Q: Is this in opposition to the type special interest group? A: No. It is meant as a simple, need based solution that could easily complement the efforts by that group. Copyright This document has been placed in the public domain. References and Footnotes [1] http://www.zope.org/Members/michel/types-sig/TreasureTrove [2] http://mail.python.org/pipermail/types-sig/2001-March/001105.html [3] http://mail.python.org/pipermail/types-sig/2001-March/001206.html [4] http://mail.python.org/pipermail/types-sig/2001-March/001223.html From paulp@ActiveState.com Wed Mar 21 02:24:31 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Tue, 20 Mar 2001 18:24:31 -0800 Subject: [Types-sig] QueryProtocol References: Message-ID: <3AB810DF.241B7D1A@ActiveState.com> "Clark C. Evans" wrote: > >... > A new built-in method, __query__ is proposed. This method has > a single argument, an interface protocol identifier, and either > returns an object supporting the given protocol, or throws an > unknown exception. What do you mean by "unknown exception". "UnknownInterface" exception? > An interface protocol identifier is a lower-case string having > a reverse DNS host name string: Why a string? What if we let an interface be any object and we used object identity to compare them? That way we don't have to introduce a new namespace into Python. I have never seen a reverse-DNS string in a Python program and I am leery of adding the concept. Maybe we will standardize the structure of interface objects and maybe we won't. Maybe we will standardize them some time in the future. I think that the addressing mechanism should be the Python namespace and not the reverse domain namespace. > In addition, a built-in query method could be introduced that > calls the __query__ method on a given object. You mean "query function." > The iterator special interest group is proposing a new built-in > called "__iter__", which could be replaced with __query__ and > a blessed interface protocol identifier of "enumerator". > Therefore calls to obj.__iter__() could be replaced with > obj.__query__("enumerator") with no semantic difference. I don't know much about __iter__ but my first instinct is to be careful there. When you return an interface, you are semantically returning a view into the object. When you return an iterator, you are semantically returning another object that happens to have special knowledge of the first. In other words, interfaces should have no state. Iterators are *all* state. :) -- Take a recipe. Leave a recipe. Python Cookbook! http://www.activestate.com/pythoncookbook From michel@digicool.com Wed Mar 21 03:04:58 2001 From: michel@digicool.com (Michel Pelletier) Date: Tue, 20 Mar 2001 19:04:58 -0800 (PST) Subject: [Types-sig] QueryProtocol In-Reply-To: <3AB810DF.241B7D1A@ActiveState.com> Message-ID: On Tue, 20 Mar 2001, Paul Prescod wrote: > > An interface protocol identifier is a lower-case string having > > a reverse DNS host name string: > > Why a string? What if we let an interface be any object and we used > object identity to compare them? That way we don't have to introduce a > new namespace into Python. I have never seen a reverse-DNS string in a > Python program and I am leery of adding the concept. I agree with this: interfaces should live in the name dot qualified namespaces that classes live in, ie, there is some absolute dotted name for the interface object ala Package.Module.Class, interfaces should be Package.Module.Interface. Giving them string names or GUID or any other identifier would be way to complex. IMO, that's what's wrong with COM. > > In addition, a built-in query method could be introduced that > > calls the __query__ method on a given object. > > You mean "query function." The way I understand the proposal, you query some interface for an object that implements an well known interface. o Where does the initial query interface come from? o Does a registry need to be kept (probably)? o What if multiple object implement the same interface? These are all fairly similar problems with COM, where Microsoft answers answer the three questions: "from us", "yes" and "they don't". I think Python should answer: "you don't need it", "no", and "no problem". -Michel From cce@clarkevans.com Wed Mar 21 03:46:25 2001 From: cce@clarkevans.com (Clark C. Evans) Date: Tue, 20 Mar 2001 22:46:25 -0500 (EST) Subject: [Types-sig] QueryProtocol In-Reply-To: <3AB810DF.241B7D1A@ActiveState.com> Message-ID: Paul, Thank you for your feedback. > "Clark C. Evans" wrote: > > A new built-in method, __query__ is proposed. This method has > > a single argument, an interface protocol identifier, and either > > returns an object supporting the given protocol, or throws an > > unknown exception. > > What do you mean by "unknown exception". "UnknownInterface" exception? I was vague here beacuse I didn't know what to do. Perhaps "None" should be returned? > > An interface protocol identifier is a lower-case string having > > a reverse DNS host name string: > > Why a string? What if we let an interface be any object and we used > object identity to compare them? That way we don't have to introduce a > new namespace into Python. I have never seen a reverse-DNS string in a > Python program and I am leery of adding the concept. Ok. This can be changed to "any object". It would definately simplify the PEP. > Maybe we will standardize the structure of interface objects and maybe > we won't. Maybe we will standardize them some time in the future. I > think that the addressing mechanism should be the Python namespace and > not the reverse domain namespace. I'm just weary about coupling this proposal with one that requires an interface object. > > In addition, a built-in query method could be introduced that > > calls the __query__ method on a given object. > > You mean "query function." How about "query object" > > The iterator special interest group is proposing a new built-in > > called "__iter__", which could be replaced with __query__ and > > a blessed interface protocol identifier of "enumerator". > > Therefore calls to obj.__iter__() could be replaced with > > obj.__query__("enumerator") with no semantic difference. > > I don't know much about __iter__ but my first instinct is to be careful > there. When you return an interface, you are semantically returning a > view into the object. When you return an iterator, you are semantically > returning another object that happens to have special knowledge of the > first. In other words, interfaces should have no state. Iterators are > *all* state. :) An iterator is a "view" of a particular object. The state the view must maintain depends upon the requirements. I don't see any semantic difficulty here at all. Clark From cce@clarkevans.com Wed Mar 21 03:50:41 2001 From: cce@clarkevans.com (Clark C. Evans) Date: Tue, 20 Mar 2001 22:50:41 -0500 (EST) Subject: [Types-sig] QueryProtocol In-Reply-To: Message-ID: On Tue, 20 Mar 2001, Michel Pelletier wrote: > > Why a string? What if we let an interface be any object and we used > > object identity to compare them? That way we don't have to introduce a > > new namespace into Python. I have never seen a reverse-DNS string in a > > Python program and I am leery of adding the concept. > > I agree with this: interfaces should live in the name dot qualified > namespaces that classes live in, ie, there is some absolute dotted name > for the interface object ala Package.Module.Class, interfaces should be > Package.Module.Interface. I'm just trying to keep the coupling very loose so that it can be fixed up later. > Giving them string names or GUID or any other identifier would be way to > complex. IMO, that's what's wrong with COM. Ok, this is three thus far that things that a string here doesn't work. In an upcoming revision this will be fixed. > > > In addition, a built-in query method could be introduced that > > > calls the __query__ method on a given object. > > > > You mean "query function." > > The way I understand the proposal, you query some interface for an object > that implements an well known interface. > > o Where does the initial query interface come from? > o Does a registry need to be kept (probably)? > o What if multiple object implement the same interface? > > These are all fairly similar problems with COM, where Microsoft answers > answer the three questions: "from us", "yes" and "they don't". > > I think Python should answer: "you don't need it", "no", and "no problem". Agreed. Although the proposal is on the surface similar to Microsoft's QueryInterface, it is very different in this respect, and many other respects. Thank you so much for taking the time to review. I'll be incorporating your comments shortly. Best, Clark From cce@clarkevans.com Wed Mar 21 09:30:16 2001 From: cce@clarkevans.com (Clark C. Evans) Date: Wed, 21 Mar 2001 04:30:16 -0500 (EST) Subject: [Types-sig] QueryProtocol Message-ID: Paul and Michel, Thank you both very much for your thoughtful feedback. I've made significant changes, many based on your comments. I'd very much apprechiate further consideration. Thank you! ;) Clark P.S. I am particularly hesitant about the baseclass matching mechanism, however, when you move to classes as an identifier, I don't think much choice exists. Yes? ---------- Forwarded message ---------- Date: Wed, 21 Mar 2001 04:23:29 -0500 From: Clark C. Evans To: python-list@python.org Newsgroups: comp.lang.python Subject: Yet Another PEP: Interface Adapter Mechanism __adapt__ Thank you all for your feedback and support with the first pass of this PEP. Below is a second attempt. The context and motivation sections are identical. Here are the major changes: * __query__ is renamed __adapt__ (adapter pattern) * __adapt__ now takes a class instance (instead of a DNS string) * __adapt__ now returns None if the lookup was unsuccessful * straw-man adapt function is detailed * the title of this PEP is updated respectively Regards, Clark Evans PEP: XXX Title: Interface Adapter Mechanism Version: $Revision$ Author: Clark Evans Python-Version: 2.2 Status: Draft Type: Standards Track Created: 21-Mar-2001 Updated: 22-Mar-2001 Summary This paper asserts that "interface typing" can be carved into two separable concerns, that of (a) protocol, which is all about behavior/expectations, and that of (b) signature which is about method existence and argument type checking. This proposal puts forth a declarative method for interface protocol discovery that could be orthogonal and complementary to a more signature based approach as being developed by the types special interest group. The proposal is modeled after the adapter pattern as described design patterns book by Gamma, Helm, Johnson and Vlissides. Context Python is a very dynamic language with powerful introspection capabilities. However, it has yet to formalize an interface or abstract type checking mechanism. A consensus for a user defined type system may be far off into the future. Currently, existence of particular methods, particularly those that are built-in such as __getitem__, is used as an indicator of support for a particular interface. This method may work for interfaces blessed by GvR, such as the new enumerator interface being proposed and identified by a new built-in __iter__. However, this current method does not admit an infallible way to identify interfaces lacking a built-in method. Motivation In the recent type special interest group discussion [1], there were two complementary quotes which motivated this proposal: "The deep(er) part is whether the object passed in thinks of itself as implementing the Foo interface. This means that its author has (presumably) spent at least a little time about the invariants that a Foo should obey." GvR [2] and "There is no concept of asking an object which interface it implements. There is no "the" interface it implements. It's not even a set of interfaces, because the object doesn't know them in advance. Interfaces can be defined after objects conforming to them are created." -- Marcin Kowalczyk [3] The first quote focuses on the intent of a class, including not only the existence of particular methods, but more importantly the call sequence, behavior, and other invariants. Where the second quote focuses on the type signature of the class. These quotes motivate a distinction between interface as a "declarative, I am a such-and-such" construct, as opposed to a "descriptive, It looks like a such-and-such" mechanism. Furthermore, it is clear that both aspects of interface are important, if not completely orthogonal and complementary. For purposes of this proposal, the word "protocol" is aligned with Guido's deep issue, and "signature" with Marcin's vision of a type system. In this way, we clearly demarcate the two attitudes of the word "interface" in a type system. Clearly, detailing an exhaustive method of interface signatures is very difficult problem, especially for a dynamic language like Python where the interface signature for a class may change over time. However, a simple declarative mechanism to inquire which interface protocols an object supports at a given time is relatively straight forward and could bring immediate benefit. Details A new built-in method, __adapt__ is proposed. This method has a single argument, an adapter identifier, and either returns an object supporting the given protocol or it returns None. As of this PEP, an adapter identifier may only be a class object. Future PEPs may expand the scope to allow other types of objects and will detail how they are treated. Further. a built-in function, adapt is suggested. This function takes two arguments, an object and an identifier. Initially, this function calls the __adapt__ method of the object, passing in the identifier. If the __adapt__ method provides an object, then this is returned. Otherwise, if the __adapt__ method is not found, or if the __adapt__ method returns None, then the objects class ancestry is checked to see if the identifier is a member. If so, then the object itself is returned. Otherwise, if no other PEPs apply, then None is return value of the adapt function. Following is a sample implementation of this built-in. def adapt(obj,ident, options = None): global check_base def check_base(bas,cmp): if bas == cmp: return 1 for base in bas.__bases__: if check_base(base,cmp): return 1 return 0 if hasattr(obj, '__adapt__'): retval = obj.__adapt__(ident) if retval == None: if check_base(obj.__class__,ident): retval = obj # options flag used to enable: # - reverse lookup via ident (future PEP) goes here? # - signature based lookup (future PEP) goes here? # - automatic signature checking (future PEP) goes here? return retval Example Usage >>> class KnightsWhoSayNi: pass >>> class EggsOnly: # an unrelated class/interface def eggs(self,str): print "eggs!" + str >>> class HamOnly: # used as an interface, no inhertance def ham(self,str): pass def _bugger(self): pass # irritating a private member >>> class SpamOnly: # a base class, inheritance used def spam(self,str): print "spam!" + str >>> class EggsSpamAndHam (SpamOnly): def ham(self,str): print "ham!" + str def __adapt__(self,cls): if cls == HamOnly: return self # HamOnly implicit, no _bugger if cls == EggsOnly: return EggsOnly() # Knows how to create the eggs! return None >>> import adapter.example >>> from adapter import adapt >>> x = adapter.example.EggsSpamAndHam() >>> adapt(x,adapter.example.SpamOnly).spam("Ni!") spam!Ni! >>> adapt(x,adapter.example.EggsOnly).eggs("Ni!") eggs!Ni! >>> adapt(x,adapter.example.HamOnly).ham("Ni!") ham!Ni! >>> adapt(x,adapter.example.EggsSpamAndHam).ham("Ni!") ham!Ni! >>> adapt(x,adapter.example.KnightsWhoSayNi).spam("Ni!") Traceback (innermost last): File "", line 1, in ? AttributeError: 'None' object has no attribute 'spam' Relationships To Iterator Proposal: The iterator special interest group is proposing a new built-in called "__iter__", which could be replaced with __adapt__ if an an Interator class is introduced. Following is an example. >>> from adapter import adapt >>> import adapter.example >>> x = adapter.example.IteratorTest(3) >>> iter = adapt(x,adapter.example.Iterator) >>> iter.next() 1 >>> iter.next() 2 >>> iter.next() Traceback (innermost last): File "", line 1, in ? File "c:\work\adapter\example.py", line 39, in next return Iterator.next(self) File "c:\work\adapter\example.py", line 24, in next raise IndexError IndexError: Relationships To Microsofts Query Interface: Although this proposal may sounds similar to Microsofts QueryInterface, it differs by a number of aspects. First, there is not a special "IUnknown" interface which can be used for object identity, although this could be proposed as one of those "special" blessed interface protocol identifiers. Second, with QueryInterface, once an object supports a particular interface it must always there after support this interface; this proposal makes no such guarantee, although this may be added at a later time. Third, implementations of Microsofts QueryInterface must support a kind of equivalence relation. By reflexive they mean the querying an interface for itself must always succeed. By symmetrical they mean that if one can successfully query an interface IA for a second interface IB, then one must also be able to successfully query the interface IB for IA. And finally, by transitive they mean if one can successfully query IA for IB and one can successfully query IB for IC, then one must be able to successfully query IA for IC. Ability to support this type of equivalence relation should be encouraged, but may not be possible. Further research on this topic (by someone familiar with Microsoft COM) would be helpful in further determining how compatible this proposal is. Backwards Compatibility There should be no problem with backwards compatibility. Indeed this proposal, save an built-in adapt() function, could be tested without changes to the interpreter. Future Compatibility It appears that this proposal could be implemented orthogonal to the protocol checking system being constructed [4] by Paul Prescod and company. In particular, a new PEP could be added which allows for Interface objects to be used as the adapter identifier. Then, the __check__ method could be used within the adapt() method. It is less clear to me how this proposal would work with the the more ambitious signature declaration and checking approach[5] taken by Michel Pelletier. This requires further investigation. Questions and Answers Q: Why was the name changed from __query__ to __adapt__ ? A: It was clear that significant QueryInterface assumptions were being laid upon the proposal, when the intent was more of an adapter. Of course, if an object does not need to be adapted then it can be used directly and this is the basic premise. Q: This is just a type-coercion proposal. A: No. Certainly it could be used for type-coercion, such coercion would be explicit via __adapt__ or adapt function. Of course, if this was used for iterator interface, then the for construct may do an implicit __adapt__(Iterator) but this would be an exception rather than the rule. Q: Why did the author write this PEP? A: He wanted a simple proposal that covered the "deep part" of interfaces without getting tied up in signature woes. Also, it was clear that __iter__ proposal put forth is just an example of this type of interface. Further, the author is doing XML based client server work, and wants to write generic tree based algorithms that work on particular interfaces and would like these algorithms to be used by anyone willing to make an "adapter" having the interface required by the algorithm. Q: Why not call this __queryinterface__ ? A: Too close to Microsofts QueryInterface, especially given the semantic differences which may not be reconcilable. Q: Is this in opposition to the type special interest group? A: No. It is meant as a simple, need based solution that could easily complement the efforts by that group. Q: Why was the identifier changed from a string to a class? A: This was done on Michel Pelletiers suggestion. This mechanism appears to be much cleaner than the DNS string proposal, which caused a few eyebrows to rise. Q: Why not allow any object to be an identifier, why just classes? A: It would be hard to get forward compatibility with new behaviors described in other PEPs. Q: It seems that a reverse lookup could be used, why not add this? A: There are many other lookup and/or checking mechanisms that could be used here. However, the goal of this PEP is to be small and sweet ... having any more functionality would make it more objectionable to some people. However, this proposal was designed in large part to be completely orthogonal to other methods, so these mechanisms can be added later if needed Copyright This document has been placed in the public domain. References and Footnotes [1] http://www.zope.org/Members/michel/types-sig/TreasureTrove [2] http://mail.python.org/pipermail/types-sig/2001-March/001105.html [3] http://mail.python.org/pipermail/types-sig/2001-March/001206.html [4] http://mail.python.org/pipermail/types-sig/2001-March/001223.html ----------------------------------------------------- adapter/__init__.py ----------------------------------------------------- def adapt(obj,cls, options = None): global check_base def check_base(bas,cmp): if bas == cmp: return 1 for base in bas.__bases__: if check_base(base,cmp): return 1 return 0 if hasattr(obj, '__adapt__'): retval = obj.__adapt__(cls) if retval == None: if check_base(obj.__class__,cls): retval = obj # options flag used to enable: # - reverse lookup via cls (future PEP) goes here? # - signature based lookup (future PEP) goes here? # - automatic signature checking (future PEP) goes here? return retval ------------------------------------------------------ adapter/example.py ------------------------------------------------------ class EggsOnly: # an unrelated class/interface def eggs(self,str): print "eggs!" + str class HamOnly: # used as an interface, no inhertance def ham(self,str): pass def _bugger(self): pass # irritating a private member class SpamOnly: # a base class, inheritance used def spam(self,str): print "spam!" + str class Intermediate (SpamOnly): pass class EggsSpamAndHam (Intermediate): def ham(self,str): print "ham!" + str def __adapt__(self,cls): if cls == HamOnly: return self # implements HamOnly implicitly, no _bugger if cls == EggsOnly: return EggsOnly() # Knows how to create the eggs! return None class Iterator: def next(self): raise IndexError class IteratorTest: def __init__(self,max): self.max = max def __adapt__(self,cls): if cls == Iterator: class IteratorTestIterator(Iterator): def __init__(self,max): self.max = max self.count = 0 def next(self): self.count = self.count + 1 if self.count < self.max: return self.count return Iterator.next(self) return IteratorTestIterator(self.max) return None From cce@clarkevans.com Wed Mar 21 10:09:13 2001 From: cce@clarkevans.com (Clark C. Evans) Date: Wed, 21 Mar 2001 05:09:13 -0500 (EST) Subject: [Types-sig] A type checking system in Python Message-ID: This was a very insightful post on the main list. ---------- Forwarded message ---------- Date: Wed, 21 Mar 2001 09:15:50 GMT From: Huaiyu Zhu Type checking system in Python Classes implement a hierarchy of inheritance of attributes (variables and methods). They are useful for implementation modularization. There is another type of modularization that is similar in nature, but is independent of classes. This is the hierarchy of polymorphic interfaces. Python supports polymorphism in a very liberal way: if it walks like a duck and quacks like a duck, then it must be a duck. The archetypical examples is that one can usually substitute a writable file with a user defined class object that happens to have a write() methods with appropriate property. In many programs it is necessary to ensure that a certain objects has certain properties. But how can we express the idea that a function expect a file-like object, while that can mean an endless list of things, many have not yet been defined? One solution promoted by some other languages is to define some abstract classes for type checking purpose, and require them to be inherited by any class that behaves like them. This is like what interfaces do in Java. This solution is not satisfactory because - It breaks polymorphism. One has to harness the property at the time the classes are defined, instead of when they are used. For example, if an application requires a property of "ReadableFile" which is satisfied by sys.stdin, it is difficult to retrofit the definition of sys.stdin to be an instance of such. - It mixes two incompatible inheritances. For example, suppose that class A has method A.m() but its subclass B overrides it with B.m(). The property A satisfies is not necessarily satisfied by B. >From the above it is clear that there is a useful conceptual hierarchy that is independent of the class hierarchy. For lack of a better name let us just call them Types. The overridability means that classes do not imply Types, and the polymorphism means that Types do not imply classes. Their main conceptual difference is that - classes embody implementations - Types embody requirements Another way to describe the difference is - classes are inherited by specialization - Types are inherited by abstraction For example, if we specify that A is a superclass of B, this does not change anything about A, but it does mean that B will have all the attributes of A. In contrast, if we specify that A is a superType of B, this does not change anything about B, but it does mean that objects accepted as B must now be accepted as A as well. As a concrete example, consider a function expecting a Number as an argument. If we declare that 2.3 is a Float, and Float is a special case of Number, then the function should accept 2.3 as argument. Therefore, Types should have their own hierarchy, possibly with its own syntax, that is independent of classes. Since Python is a very dynamically typed system, it is also desirable to make the type checking as flexible as possible, and completely voluntary, while being able to deliver the same level of guarantees that a strongly and statically typed language would give. Type can also be used to express properties like Immutable, which could only be guaranteed by a compiler. These properties will also be very useful for compiler optimization. All said, there is a role for classes to play in this affair - It is possible to fake such a Type system using a class, with some caveats: - It is dynamic, and very inefficient at run time - It is dynamic, and can be circumvented in various ways. Nevertheless, this dynamical type-checking does provide a fine illustration of the concepts discussed above. If and when they can be coded in compiler it will be of much greater use. ================================================================== #!/usr/bin/env python # $Id: test.py,v 1.1.1.1 2000/08/29 00:38:54 hzhu Exp $ """ A simulated type checking system Types are inherited by abstraction """ class Type: def __init__(self, name): "Define a new abstract Type" self.name = name self.types = [] self.classes = [] self.subs = [] self.attrs = [] self.objs = [] def include(self, obj): "Check to see if object is of my Type" # If it is explicitly included objects for o in self.objs: if obj == o: return 1 # If it is of the right type or class if type(obj) is type(Type): for c in self.classes: if isinstance(obj, c): return 1 else: for t in self.types: if type(obj) == t: return 1 # If it is one of our sub-Type for s in self.subs: if s.include(obj): return 1 else: return 0 def satisfiedby(self, obj): """Check to see if object has all the required attributes Is this necessary? """ for a in self.attrs: if not hasattr(obj, a): print "%s has no attr %s" %(`obj`, a) raise AssertionError def add_type(self, t): if t not in self.types: self.types.append(t) def add_class(self, c): if c not in self.classes: self.classes.append(c) def add_sub(self, s): if s not in self.subs: self.subs.append(s) def add_attr(self, m): if m not in self.attrs: self.attrs.append(m) def add_obj(self, o): if o not in self.objs: self.objs.append(o) def add_typeof(self, o): self.add_type(type(o)) def add_classof(self, o): self.add_class(o.__class__) def add_attrof(self, o): for attr in dir(o): self.add_attr(attr) def __repr__(self): return "Type(%s)" % `self.name` def require(x, t): try: assert t.include(x) except AssertionError: print "%s is not of %s" % (`x`, t) raise t.satisfiedby(x) def require_either(x, list): for t in list: if t.include(x): t.satisfiedby(x) return else: print "%s is not of any Type in %s" % (`x`, list) raise AssertionError def require_all(x, list): for t in list: require(x,t) #------------------------------------------------------------------ String = Type("String") String.add_typeof("") Integer = Type("Integer") Integer.add_typeof(1) Integer.add_typeof(1L) Number = Type("Number") Number.add_sub(Integer) Float = Type("Float") Float.add_typeof(0.0) Number.add_sub(Float) File = Type("File") File.add_attr("close") #------------------------------------------------------------------ if __name__ == "__main__": def f(x): require(x, Number) print x*2 f(1) f(2.3) def f(x): require_either(x, (Number, String)) print x*2 f("2") import sys File.add_obj(sys.stdout) require(sys.stdout, File) require_either("2", (Number, String)) #require_all("2", (Number, String)) -- http://mail.python.org/mailman/listinfo/python-list From robin.thomas@starmedia.net Wed Mar 21 11:31:50 2001 From: robin.thomas@starmedia.net (Robin Thomas) Date: Wed, 21 Mar 2001 06:31:50 -0500 Subject: [Types-sig] A type checking system in Python In-Reply-To: Message-ID: <4.3.1.2.20010321062815.0223ec30@exchange.starmedia.net> At 05:09 AM 3/21/01 -0500, Clark C. Evans wrote: >This was a very insightful post on the main list. Yes, I liked it a lot. I don't see the need for the extensiveness of the example code to be *built-in* features of Python. Only the feature for checking/likening/adapting an object with some identifier/signifier is needed to be built-in, IMO. The rest can come later after everyone plays with the feature in real-world code. -- Robin Thomas Engineering StarMedia Network, Inc. robin.thomas@starmedia.net From robin.thomas@starmedia.net Wed Mar 21 11:33:28 2001 From: robin.thomas@starmedia.net (Robin Thomas) Date: Wed, 21 Mar 2001 06:33:28 -0500 Subject: [Types-sig] QueryProtocol In-Reply-To: Message-ID: <4.3.1.2.20010321052249.00d6d1b0@exchange.starmedia.net> At 04:30 AM 3/21/01 -0500, Clark C. Evans wrote: >Thank you all for your feedback and support with the first >pass of this PEP. Below is a second attempt. The context and >motivation sections are identical. Here are the major changes: > > * __query__ is renamed __adapt__ (adapter pattern) > * __adapt__ now takes a class instance (instead of a DNS string) > * __adapt__ now returns None if the lookup was unsuccessful > * straw-man adapt function is detailed > * the title of this PEP is updated respectively This is very good. It actually can assimilate my earlier proposal for abstract type checking via a __type__ method. I also see that adapt() works to assimilate isinstance(), and probably could assimilate issubclass(). I support that work, but I see that it gives some complexity to the meaning of the return value -- in adapter-like cases, you want to return self or an adapter of self; in test-like cases, you want to return a boolean. I also think that is very cool. I like "adapt" better than "query". I also think that this function is a candidate to be an operator in the future. The two quotes you mention in the PEP, to me at least, boil down to: Quote 1: "The object may have the answer, so ask it about the identifier." Quote 2: "The identifier may have the answer, so ask it about the object." This makes me think of adapt() as a binary op just as +, ^, divmod, etc. A word that works nicely as an operator name, especially a simple word that novices can grasp quickly, would be ideal as a name for this new feature. Suggestions: 1) "like" like(obj, ident, options=None) # future operator "like" and "not like" a like b # equiv to like(obj,ident) a not like b # equiv to not like(obj,ident,"test") 2) "is" with builtin isa() or even just adapt() a is b # adapt(a,b) as adapt written below a is not b # not adapt(a,b,"test") adapt(obj, ident, options=None) Suggested impls for adapt: # the like version # ! uses builtins/ops as shorthand def like(obj, ident, options=None): # if same object, TRUE if obj is ident: return 1 obj_type = type(obj) # if one is the concrete type of another, # or if they have the same concrete type, TRUE if obj_type is ident: return 1 ident_type = type(obj) if obj_type is ident_type or obj is ident_type: return 1 # seems that isinstance() and issubclass() can be implemented # here if we return true cases and let the false cases fall # through to the __like__ check. retval = 0 if ident_type is ClassType: # isinstance if obj_type is InstanceType and isinstance(obj, ident): return 1 # issubclass elif obj_type is ClassType and issubclass(obj, ident): return 1 # __like__, if a "builtin", is actually in the type methods # table, not a real named attribute. obj.__like__ is just # the way for instances to emulate the builtin method. Right? # So the code below is pseudo-code for what would happen # at the C level. # try the obj's type first if obj_type.slot('__like__'): retval = obj_type.slot('__like__')(obj, ident) # else try the ident if ident_type.slot('__like__'): retval = ident_type.slot('__like__')(ident, obj) # options hook (if still needed) return retval And the implementations for some built-in types: InstanceType: if hasattr(self, "__like__"): return self.__like__(ident) All the others are wide open. If the types module gets some abstract type objects like Mapping or Sequence or even things like Mutable, the __like__ slot exists to implement a type hierarchy. I considered using ClassType's slot function to implement isinstance() and issubclass(), or at least issubclass(). Maybe that's still a good idea. Clark, are our thoughts synchronizing in any way? How can I assist in PEP or implementation? -- Robin Thomas Engineering StarMedia Network, Inc. robin.thomas@starmedia.net From ping@lfw.org Wed Mar 21 13:06:18 2001 From: ping@lfw.org (Ka-Ping Yee) Date: Wed, 21 Mar 2001 05:06:18 -0800 (PST) Subject: [Types-sig] A type checking system in Python In-Reply-To: Message-ID: On Wed, 21 Mar 2001, Clark C. Evans wrote: > This was a very insightful post on the main list. > > ---------- Forwarded message ---------- > Date: Wed, 21 Mar 2001 09:15:50 GMT > From: Huaiyu Zhu > > Type checking system in Python > > Classes implement a hierarchy of inheritance of attributes (variables and > methods). They are useful for implementation modularization. > > There is another type of modularization that is similar in nature, but is > independent of classes. This is the hierarchy of polymorphic interfaces. Indeed. I think it's worthwhile for anyone who's seriously thinking about these issues to have a look at Sather's type system: http://www.icsi.berkeley.edu/~sather/ In particular, see: http://www.icsi.berkeley.edu/~sather/Documentation/LanguageDescription/contents.html (sections 4 and 5) Sather separates subtyping (= substitutability) from implementation sharing. It also permits supertyping, i.e. declaring something to be a supertype of another. (As a side note, Sather has iterators and uses the keyword "yield".) -- ?!ng Happiness comes more from loving than being loved; and often when our affection seems wounded it is is only our vanity bleeding. To love, and to be hurt often, and to love again--this is the brave and happy life. -- J. E. Buchrose From sverker.is@home.se Wed Mar 21 16:55:57 2001 From: sverker.is@home.se (Sverker Nilsson) Date: Wed, 21 Mar 2001 17:55:57 +0100 Subject: [Types-sig] Interface PEP References: <3AAFCBDF.39A0735D@home.se> <3AB00D5A.453E7BCF@home.se> Message-ID: <3AB8DD06.6CD4206@home.se> Marcin 'Qrczak' Kowalczyk wrote: > > Thu, 15 Mar 2001 01:31:22 +0100, Sverker Nilsson pisze: > > > What not only many but perhaps all people would agree on, including > > me, is that the builtin types or classes should be unified with > > user-defined classes in this way: So that user-defined classes can > > inherit from the classes that the objects with builtin types have. > > I agree. Ok. > > > That doesn't necessary mean unifying types with classes. It could just > > as well mean that we defined the classes of the builtin objects. > > Why not to unify then? I'm not sure how that will work or what is really meant. But I suppose that the unification you are thinking of, would make types and classes the same. In every context you say type you could likewise say class and vice versa. I don't see how this could be done or how useful it is. How about the type() builtin. I assume we can't remove it or rename it to class() because it may break too much code. What should it return for an instance of a user-defined class then? If it still returned InstanceType, I don't see that the type and classes have been logically unified. Because what would InstanceType be. If all types would be classes, InstanceType would also be a class. But it would be separate from the class of the object itself. That is in contrast with type() of a builtin object which would be it's class. For other objects it would be InstanceType. So there is no unification in this respect. On the other hand if type() of user-defined objects would return its class, it would break unacceptably much code, I (strongly) believe. Therefore, it seems to me type and class can not be really unified. However, we can define the classes of builtin objects/types, without the above suggested conceptual problems. I said previously we could have e.g. BuiltinClasses.List etc. Now I have seen that Guido has previously suggested something similar in a mail on Python-Dev. http://mail.python.org/pipermail/python-dev/2000-November/010417.html Guido: """ As long as we're proposing hacks like this that don't allow smooth subclassing yet but let you get at least some of the desired effects, I'd rather propose to introduce some kind of metaclass that will allow you to use a class statement to define this. Thinking aloud: import types filemetaclass = metaclass(types.FileType) class myfile(filemetaclass): """ Granted, he seem to regard this mostly as a temporary solution, but I don't see why it can't be part of the permanent solution. > > I understand that there are technical reasons, like the speed > resulting from avoiding looking up operations by name, or the fact [snip yes I am not referring to technical implementation] > But from the point of users of the language there is no reason why > files should be objects of FileType and not objects of InstanceType > with class File. It's an artifact of the implementation. Are you saying that type() should return InstanceType for ALL types, builtin or otherwise? That would break a lot of code! Totally unacceptable, I would say. > > > Why would it be cleaner to unify type and class, given the many > > variants that exist, that you describe (some of) yourself below? > > Because concepts for which many other languages use terms "type" > and "class" are more different than concepts for which Python uses > these terms (except C++, but let's not follow this crazy language). > > These words have different meanings in those languages and whether > types and classes can be unified in Python is independent from whether > they can be unified in Haskell. It's just a terminology clash. Not going into this much deeper for now. Both interface, type and class are overloaded terms. Builtin Python types happen to specify specific implementations. I still don't see why we can't have something more we call types, user define types, that happen to specify another implementation (such as the implementation shared by all InstanceType objects) but mostly are used to specify a protocol/interface. It's about 'type checking', after all. [snip] > [ About Haskell static type system, I wrote: ] > > And I read some paper that showed how impossible it was to define > > a natural join... in some context. > > I don't know what is natural join. I'll try to explain the problem - maybe it is of some interest for the Python efforts too. [ Disclaimer: I don't have the original paper handy so this is guesses from memory, I may be missing their point ;-/] >From p.82 in "Database System Concepts", Silberschatz et al, 3:d edition: The natural-join operation forms a Cartesian product of its two arguments, performs a selection forcing equality on those attributes that appear in both relation schemas, and finally removes duplicate attributes. I suppose it's the "removes duplicate attributes" that would create some of the (worst) problems... but on second thought it seems more complicated than that. To illustrate... a Python natural-join procedure might look like: def natural_join((schema_a, relation_a), (schema_b, relation_b)): # schema_[ab] are tuples of attribute names # relation_[ab] are list of tuples of attribute values # each tuple in relation_[ab] has same length as schema_[ab] # The types of attribute values can be different for each attribute. # The types of attribute values should support comparison at # least for equality; support for ordering (<) or hashing will # allow for faster algorithms to be applied. This, however, only # really applies to the attribute values that are really compared, # that is, the attributes that have names that occur in both # schema_a and schema_b. # The type of attribute names should support comparison for equality. # (Specifying anything more would not speed up anything significantly # except for pathological cases.) # Return: (schema_ret, relation_ret), where: # # schema_ret is a tuple of all unique attribute names from schema_ab # relation_ret is a list of tuples with values corresponding to schema_ret # # Typecheck and implementation left as an excercise The type of the parameter objects are dependent in complicated ways. If we want to 'automate' the dynamic type check, it won't suffice to give a type after each parameter. We need to give a type for all the parameters together, or suitable combinations of them. Paul Prescod's system has a dict with types of parameters separately specified. When the parameters are dependent, however, it seems we need something more general. We might want type check functions that take any combination of parameters. Doing the check dynamically should then be possible, 'automated' or not, given suitable user-defined type-check functions. But statically I don't know if the compiler can deduce anything useful in complicated cases like this. This is not a critique of Haskell per se - this check may well be impossible to do statically at all... Although on the other hand that is not the impression I remember from the original paper that was about Haskell. Maybe I misunderstood something, maybe there could be some practical way to do typing like this statically. [snip] > Types define the representation. Interfaces define the usage. Classes > are somewhat between. Not necessarily. I have seen some other views expressed. > In dynamically typed languages they are much > closer to types, because usage is not checked wrt. declared classes - > in Python subclassing is only used to inherit behavior, not to create > subtypes. > > Sorry, unifying types and interfaces while leaving classes alone is > complete nonsense. Please, I think blank statements like that don't contribute well to a discussion where we are considering various ideas in an open manner... with various degrees of expertise of course but trying our best. That said, I believe it would be good to have a single way to refer to what we put as the type-indication. If we have this syntax for example: def f(x:y) What's the y? Most languages call it a type, I think. However, as I found myself indicating in the natural-join example, it will not suffice to specify the types separately for each parameter. Something more generally is needed. Only because it's more general doesnt mean we can't consider it to be a type though, IMHO. In Haskell you could specify interfaces AND types, when defining the type of an object. They are specified in different ways. You can say, if I remember correctly, for example: f :: Integral a, Integral b => (a->b)->Tree a->Tree b where Integral is a class (interface) and Tree is a parameterized type. Type specifications can get long. But you can't define a type that contains all the above - this is not allowed, IIRC: type atype = Integral a, Integral b => (a->b)->Tree a->Tree b You have to repeat all of 'Integral a, Integral b => (a->b)->Tree a->Tree' for each definition of functions of the same type. That's an unnecessary restriction, I think. Even though it may be necessary in the context of the specific Haskell system of types, it should not be necessary in general to keep interfaces and types that rigidly separated. tongue-in-cheek-ly yours, Sverker From paulp@ActiveState.com Wed Mar 21 18:04:58 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Wed, 21 Mar 2001 10:04:58 -0800 Subject: [Types-sig] QueryProtocol References: Message-ID: <3AB8ED4A.66C4B961@ActiveState.com> "Clark C. Evans" wrote: > >... > > I was vague here beacuse I didn't know what to do. > Perhaps "None" should be returned? That would probably be easiest. If the caller wants to require an interface, they can throw the exception themself. > ... > > > In addition, a built-in query method could be introduced that > > > calls the __query__ method on a given object. > > > > You mean "query function." > > How about "query object" If you are talking about a built-in function, you should call it a built-in function. like hasattr, getattr, etc. >... > > I don't know much about __iter__ but my first instinct is to be careful > > there. When you return an interface, you are semantically returning a > > view into the object. When you return an iterator, you are semantically > > returning another object that happens to have special knowledge of the > > first. In other words, interfaces should have no state. Iterators are > > *all* state. :) > > An iterator is a "view" of a particular object. The state > the view must maintain depends upon the requirements. > I don't see any semantic difficulty here at all. Here's my definition of "view". If x and y are views over the same object then at any time I should be able to call x.foobar() or y.foobar() and expect to get back the same result. If x.foobar() can return a different result than y.foobar() then the objects are really different objects, not just views into a single object. -- Take a recipe. Leave a recipe. Python Cookbook! http://www.activestate.com/pythoncookbook From paulp@ActiveState.com Wed Mar 21 18:15:22 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Wed, 21 Mar 2001 10:15:22 -0800 Subject: [Types-sig] QueryProtocol References: Message-ID: <3AB8EFBA.B7E8253C@ActiveState.com> Michel Pelletier wrote: > >... > > The way I understand the proposal, you query some interface for an object > that implements an well known interface. As I understand it, you query an *object* for another object that implements an interface. You get back an object that conforms to the interface or None. There is no concept of a vtable-style interface object unless you want it. > o Where does the initial query interface come from? It's just any Python object. > o Does a registry need to be kept (probably)? I think we all agree now that the answer is "no". > o What if multiple object implement the same interface? I don't think that is a problem with COM, XPCOM or this proposal. The ability for multiple objects to implement the same interface is the whole point of interfaces in any of these systems! -- Take a recipe. Leave a recipe. Python Cookbook! http://www.activestate.com/pythoncookbook From paulp@ActiveState.com Wed Mar 21 19:13:00 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Wed, 21 Mar 2001 11:13:00 -0800 Subject: [Types-sig] A type checking system in Python References: Message-ID: <3AB8FD3C.DA339015@ActiveState.com> "hzhu@users.sourceforge.net" wrote: > > ... > > One solution promoted by some other languages is to define some abstract > classes for type checking purpose, and require them to be inherited by any > class that behaves like them. This is like what interfaces do in Java. > This solution is not satisfactory because > > - It breaks polymorphism. One has to harness the property at the time the > classes are defined, instead of when they are used. For example, if an > application requires a property of "ReadableFile" which is satisfied by > sys.stdin, it is difficult to retrofit the definition of sys.stdin to be > an instance of such. It is too strong to say it breaks polymorphism. It can in some cases *hamper* polymporhism. But Java is a fundamentally polymorphic language, as are all other object oriented languages I have used. Polymorphism is the central point of OO. Even languages that do not support either encapsulation or inheritance support polymorphism. > - It mixes two incompatible inheritances. For example, suppose that class A > has method A.m() but its subclass B overrides it with B.m(). The property > A satisfies is not necessarily satisfied by B. You cite Java interfaces above and yet Java interfaces are *specifically designed* to separate out the two different kinds of inheritance. That's why there are interfaces which are distict from classes! > From the above it is clear that there is a useful conceptual hierarchy that > is independent of the class hierarchy. For lack of a better name let us > just call them Types. We've been calling them interfaces and/or protocols. Python already has a meaning for "type" which is somewhat confusing in this context. > - classes embody implementations > - Types embody requirements This is exactly the distinction between Java classes and interfaces. I have a prototype implementation that embodies this distinction here: http://cvs.sourceforge.net/cgi-bin/cvsweb.cgi/python/nondist/sandbox/typecheck/?cvsroot=python We've been discussing these issues in the type-sig. You are invited to join us there. -- Take a recipe. Leave a recipe. Python Cookbook! http://www.activestate.com/pythoncookbook From qrczak@knm.org.pl Wed Mar 21 20:00:19 2001 From: qrczak@knm.org.pl (Marcin 'Qrczak' Kowalczyk) Date: 21 Mar 2001 20:00:19 GMT Subject: [Types-sig] Interface PEP References: <3AAFCBDF.39A0735D@home.se> <3AB00D5A.453E7BCF@home.se> <3AB8DD06.6CD4206@home.se> Message-ID: Wed, 21 Mar 2001 17:55:57 +0100, Sverker Nilsson pis= ze: > How about the type() builtin. I assume we can't remove it or rename > it to class() because it may break too much code. What should it > return for an instance of a user-defined class then? Deprecate it. Let it behave as currently, but it has no place when types and classes are unified. Have another function which returns the unified type/class of an object. Or change it. Let it return the class object instead of InstanceType. Use from __future__ or another mechanism to make the transition smooth. I don't care. I'm not talking how to smoothly apply this unification to existing Python, but how a Python with unified types+classes would look like. > Granted, he seem to regard this mostly as a temporary solution, > but I don't see why it can't be part of the permanent solution. Because the special status of builtin types is only an artifact of the current implementation. The split doesn't carry a meaningful message. The same thing can be often done as either type or class. > Builtin Python types happen to specify specific implementations. > I still don't see why we can't have something more we call types, > user define types, that happen to specify another implementation > (such as the implementation shared by all InstanceType objects) > but mostly are used to specify a protocol/interface. Because interface is the opposite of implementation! The point in an abstract interface is that it does not say anything about the implementation. A type defines an implementation (and indirectly some interface, but parts of this interface are shared by many types and classes). A class defines an implementation (and indirectly some interface, but parts of this interface are shared by many types and classes). An interface doesn't restrict the object to have a particular type or class. It only says how the object can be used. Interfaces are not formalized in Python, but could be, and the purpose of this list is to design an interface framework and concrete interfaces in it. Type and class are almost the same concept. Interface is a completely different concept. > That said, I believe it would be good to have a single way to > refer to what we put as the type-indication. If we have this syntax > for example: >=20 > def f(x:y) >=20 > What's the y? Most languages call it a type, I think. It doesn't matter how they call it. It matters what it means. Since it determines what f assumes about x and which objects are acceptable for x, it seems natural to call it an interface. For example "sequence" and "file-like object" are already existing Python interfaces, even if not strictly defined. If f indeed requires x to be a specific type or class, and it's not enough for x to have specific methods with specific signatures and meanings, we can make an interface from a type or a class. Such interface is quite constraining: it requires an object to have the specific type or inherit from the specific class. For convenience we can allow type objects and class objects to be used as interfaces directly, instead of having to explicitly obtain the interface generated by a type or class. > In Haskell you could specify interfaces AND types, when defining > the type of an object. They are specified in different ways. You > can say, if I remember correctly, for example: >=20 > f:: Integral a, Integral b =3D> (a->b)->Tree a->Tree b >=20 > where Integral is a class (interface) and Tree is a parameterized > type. Indeed. > Type specifications can get long. But you can't define a type that > contains all the above - this is not allowed, IIRC: >=20 > type atype =3D Integral a, Integral b =3D> (a->b)->Tree a->Tree b This is because class constraints are not a part of the type. You can at most write type AType a b =3D (a->b)->Tree a->Tree b and then f :: (Integral a, Integral b) =3D> AType a b This signature of f is identical to f :: (Integral c, Integral d) =3D> AType c d Names of type variables are not a part of a complete type. It would be meaningless to have type variables in the definition which are absent in the left hand side, i.e. type Spam =3D a -> b Such type doesn't exist by itself. Type variables have a meaning only in the context of a complete type of a variable. > That's an unnecessary restriction, I think. Even though it may be > necessary in the context of the specific Haskell system of types, > it should not be necessary in general to keep interfaces and types > that rigidly separated. It's a syntactic detail. Having to express interfaces as types (as in Java and Eiffel) has more serious restrictions. It is generally wrong in its core assumptions. Some interfaces can only be considered properties of types, not of objects of these types. In Java and Eiffel you cannot say that two arguments of a function have a type which supports a sequence interface. You can say that they are both sequences, but it's not enough to be able to concatenate them (let's assume that sequences must support concatenation). You cannot concatenate a list with a tuple, or a tuple with a string. But you can concatenate two lists, or two tuples, or two strings. In general you can concatenate two objects of the same type if it's a sequence type. You cannot express this by interfaces of individual sequence objects. You cannot say that a function can be called with a list of objects of some comparable type, and will return an object of that type, no matter what type it is. You cannot say that the first argument is a mapping, and the second argument has the matching type for its keys, and the result has the matching type for its values. You cannot say that an object is both a mapping and something which can be written to a file. You cannot say that an argument is a (possibly empty) list of "statements" producing values of some type, let's call it a, and the result is a "statement" producing a list of values of type a. "Statement" means an object whose type supports the "statement" interface, call it "monad", which defines how to compose statements and how to make a trivial statement returning some value. The assertion that the function f has this type is spelled thus in Haskell: f :: Monad m =3D> [m a] -> m [a] and thus in C++ (including the comment): template