From skaller@maxtal.com.au Sun Jan 2 00:52:52 2000 From: skaller@maxtal.com.au (skaller) Date: Sun, 02 Jan 2000 11:52:52 +1100 Subject: [Types-sig] import vs include (was: RFC Comments) References: Message-ID: <386EA164.4C371705@maxtal.com.au> Greg Stein wrote: > > On Thu, 30 Dec 1999, skaller wrote: > >... > > In this 'two pass' model, it is inconsistent to > > 'import' a module in pass 1, since 'importing' a module > > requires a recursive tranlation pass involving TWO passes, > > and we know that the second pass can even involve recursive > > module execution. So it isn't _possible_ to import > > a module during pass 1. It won't work. > > Python importing does *not* allow recursive module execution. > > a.py: > import b > some_code() > > b.py: > import a > more_code() This is what I mean by a recursive import. a import b, b imports a. -- John (Max) Skaller, mailto:skaller@maxtal.com.au 10/1 Toxteth Rd Glebe NSW 2037 Australia homepage: http://www.maxtal.com.au/~skaller voice: 61-2-9660-0850 From skaller@maxtal.com.au Sun Jan 2 00:54:14 2000 From: skaller@maxtal.com.au (skaller) Date: Sun, 02 Jan 2000 11:54:14 +1100 Subject: Anti-poking lobby (was:Re: [Types-sig] type declaration syntax) References: <386165AF.F6E6BF81@maxtal.com.au> <19991230215036009.AAA186.69@max41121.izone.net.au> Message-ID: <386EA1B6.4A3A7C56@maxtal.com.au> John Machin wrote: > Once we have banned poking from outside a module, can't we fix the > presumably-few cases of missing-but-required functionality by supplying > functions? For example, > > previous_stdout = sys.divert_stdout(new_stdout_file-like_object) Of course, but the issue is compatibility with existing code. -- John (Max) Skaller, mailto:skaller@maxtal.com.au 10/1 Toxteth Rd Glebe NSW 2037 Australia homepage: http://www.maxtal.com.au/~skaller voice: 61-2-9660-0850 From sjmachin@lexicon.net Sun Jan 2 20:10:39 2000 From: sjmachin@lexicon.net (John Machin) Date: Mon, 3 Jan 2000 06:10:39 +1000 Subject: Anti-poking lobby (was:Re: [Types-sig] type declaration syntax) In-Reply-To: <386EA1B6.4A3A7C56@maxtal.com.au> Message-ID: <20000102190145574.AAA49.241@max41122.izone.net.au> > John Machin wrote: > > > Once we have banned poking from outside a module, can't we fix the > > presumably-few cases of missing-but-required functionality by > > supplying functions? For example, > > > > previous_stdout = sys.divert_stdout(new_stdout_file-like_object) > > Of course, but the issue is compatibility with > existing code. Can't we overcome such issues with much better alternatives, sniffer kits which find instances of the soon-to-be-forbidden constructs, a little marketing, and a positive attitude? From gstein@lyra.org Sun Jan 2 20:31:41 2000 From: gstein@lyra.org (Greg Stein) Date: Sun, 2 Jan 2000 12:31:41 -0800 (PST) Subject: [Types-sig] import vs include In-Reply-To: <386EA164.4C371705@maxtal.com.au> Message-ID: On Sun, 2 Jan 2000, skaller wrote: > Greg Stein wrote: > > On Thu, 30 Dec 1999, skaller wrote: > > >... > > > In this 'two pass' model, it is inconsistent to > > > 'import' a module in pass 1, since 'importing' a module > > > requires a recursive tranlation pass involving TWO passes, > > > and we know that the second pass can even involve recursive > > > module execution. So it isn't _possible_ to import > > > a module during pass 1. It won't work. > > > > Python importing does *not* allow recursive module execution. > > > > a.py: > > import b > > some_code() > > > > b.py: > > import a > > more_code() > > This is what I mean by a recursive import. > a import b, b imports a. I *know* that. I was responding to your point about recursive execution. There is no recursive *execution*. As a result of this, there is no need to define a two-pass mechanism for imports during compile-time checking. Therefore, there is no need for "include". Cheers, -g -- Greg Stein, http://www.lyra.org/ From gstein@lyra.org Sun Jan 2 21:17:42 2000 From: gstein@lyra.org (Greg Stein) Date: Sun, 2 Jan 2000 13:17:42 -0800 (PST) Subject: [Types-sig] new namespaces? no... (was: PyDL RFC 0.02) In-Reply-To: <3867FAED.BAEC3B6@prescod.net> Message-ID: On Mon, 27 Dec 1999, Paul Prescod wrote: > Greg Stein wrote: > > Windows 9x people can very well have problems. The underlying filesystem > > is still 8.3. I continued to see issues with the name mapping between long > > and short. Mostly, it appears with certain APIs and the registry. > > > > Seriously: avoid more than .3 if possible. > > Okay, .pyi will be the extension but I won't give up on the pun as the > formal name for the language without more teeth pulling (and I've just > had my wisdom's removed so my tolerance level is high). You're still referring to .pydl and .gpydl in the RFC and stuff... > > "in the future" is a *long* ways off when there hasn't been any real > > discussion on if/how to deal with the multiple namespace issue. Relying on > > a solution to appear is asking for trouble (IMO). > > It seems to me that the simplest solution is to move the "types" > namespace BEHIND the __builtin__ namespace. That doesn't resolve some of the other issues. A lot of things presume just two namespaces plus the builtins. Now you're sneaking in another namespace that must be handled. > > However: I'm still against adding a whole new namespace. I haven't seen a > > good argument for why it is needed. Can somebody come up with a concise > > rationale? > > Well there are a few issues and I admit to having not thought all of > them through completely yet: All right. I'll respond to your points here. Hopefully, I can show that a new namespace is not needed... > * importing modules are supposed to only see exported attributes. For > instance dir() should only show exported attributes. This is a big semantic change, and I don't think we should consider limiting availability like this. > * the two namespace arrangement is similar to the way that a class' > namespace is segmented from that of instances. I don't understand this one. There isn't a relation between a "typedecl namespace" and the other namespaces, akin to the class/instance relationship. > * Types are independent objects but variable declarations need to be > somehow unified with the declared objects. I agree that we need to have objects for these things (e.g. similar to the objects in my typedecl.py). Declarations can go into an attribute much like __members__ or __dict__ or whatever. I've suggested that we attach interface objects to modules and classes; those, in turn, provide the declarations for each attribute of an interface. Function objects carry their own signature, but they could also be reachable from the interface object. > * But we also need an API to query type information associated with a > name (instead of the value bound to the name) Yes, we do, but this isn't a new namespace (in terms of searching for a name's value). This is just storing a name:typedecl mapping somewhere. > * Type expressions can make forward references. So when they are > embedded in Python code we still won't think of them as ordinary > assignments. I believe the only forward reference that is possible, is that of an interface or class. I have proposed an "incomplete" interface/class for handling this problem (much like an incomplete struct in C). A separate namespace does not intrinsicly handle the forward-referencing -- it just moves the problem somewhere else. > I have not put a lot of thought into this part of the system and am open > to suggestions of how to get all of this to work. 1) Drop the concept of a separate namespace. 2) Add incomplete interfaces/classes. This means that any (dotted) name used in a type expression can be defined by the time it is used. So back to my original question: if you still think we need a new namespace (in which names are looked up as part of the standard name/value resolution), then please explain why. I don't think we need to introduce a new namespace. Cheers, -g -- Greg Stein, http://www.lyra.org/ From gstein@lyra.org Sun Jan 2 22:52:28 2000 From: gstein@lyra.org (Greg Stein) Date: Sun, 2 Jan 2000 14:52:28 -0800 (PST) Subject: [Types-sig] module's interface(s) (was: rebinding) In-Reply-To: <386C780A.3BFEE183@prescod.net> Message-ID: On Fri, 31 Dec 1999, Paul Prescod wrote: > Greg Stein wrote: > > ... > > > * module -- we must always disallow rebinding these because we don't > > > have a notion of two modules with the "same interface". Maybe in some > > > future version we could. > > > > Untrue. Ever look at the "anydbm" module and its cohorts? How about the > > DBAPI modules? > > I didn't say that there was no notion of modules with the same > interface. I said that our type declaration sub-language does not have > such a notion. Then I think we have a problem :-) We should define a way that a module can state its interface. For example, if we say that you associate an interface with a class by assigning to an __interfaces__ attribute, then I'd say we use the same mechanism at the module-level to specify the module's interface(s). Cheers, -g -- Greg Stein, http://www.lyra.org/ From gstein@lyra.org Mon Jan 3 00:26:23 2000 From: gstein@lyra.org (Greg Stein) Date: Sun, 2 Jan 2000 16:26:23 -0800 (PST) Subject: [Types-sig] Re: syntax and compile/run -time In-Reply-To: <386C79A7.CBB008E@prescod.net> Message-ID: On Fri, 31 Dec 1999, Paul Prescod wrote: > Greg Stein wrote: > > ... > > > And I put decl > > > and typedecl at the front instead of making them operators because I > > > agree with Tim Peters that we are designing a sub-language that needs to > > > be understood as being separate by virtue of being evaluated BEFORE the > > > code is executed. > > > > I disagree. Making a "sub-language" will simply serve to create something > > that is not integrated with Python. I see no reason to separate anything > > that is happening here -- that is a poor requirement/direction to take. > > Compile time stuff is inherently separate because it is *compile time > stuff*. So? The type checker uses a lot of runtime-like stuff to do its work. x = y + z That is arguably runtime, but the type checker will be handling it at compile time -- it has to understand "y" and "z" and what happens when you use the "+" operator on them, to produce a result for assignment to "x". Dividing the language into pieces is simply dividing it. I see no rational reason for doing so, but many reasons to keep everything clean and integrated. > It follows different import rules, it is evaluated in a > different namespace, and so forth. The import rules are barely different. It looks for an interface file and reads that, but there isn't any reason it couldn't also find/read/parse a module file to extract an interface. As I've attempted to explain in other notes, I don't believe a different namespace is required. > This, for instance, is not legal: > > a = doSomething() > b = typedef a It certainly can be legal. But the type-checker would say "I can't help you here." Note: you don't need the typedef in the above expression. The typedef operator simply serves to provide access to a typedecl object (and its particular construction syntax). Generally, you would only need a typedef for things such as: IntOrString = typedef Int or String ListOfInt = typedef [Int] It is also helpful to note that the type-checker has no problem with the above statements. You wouldn't need a typedef for things like: IntAlias = Int MyClassAlias = MyClass MyIfcAlias = MyInterface String = type("") In each of these cases, the LHS is assigned an object that can be used in type declarators later in the program. A name in a type declarator can refer to a base type, a class, an interface, or another type declarator. Type declarator objects are usually only used for complex typedecls -- alternates such as IntOrString, or parameterized (abstract or concrete) typedecls. Even with the above statements, the type checker can easily track them and use the names later on (again: it has to track these kinds of things just as part of its normal job). > Python programmers need to understand these sorts of things. The decl > syntax and "gpydl" semantics makes it very clear that these declarations > are *separate* and are evaluated in a different time in a different > execution context with a different namespace. I believe this distinction is unnecessary and is on the wrong track. I do not believe there is a requirement to create a partitioning of the language and its semantics. It is much better to leverage the existing semantics of Python than to create a whole new set. There are very few constructs that a person will attempt, for which we cannot track the implied type information. We can easily identify those situations and warn the user that we cannot provide a compile-time check for them. Specifically, in your example: if "a" or "b" was used in a later typedecl: def foo(x: a)->None: ... We Could issue a warning that might read, "The type information represented by is not available at compile time; compile-time checking is not enabled for parameter in the function , nor for calls to ." [ well, some suitable word-smithing is needed there :-) ... but you get the idea. ] I believe the only real time that we can't tell is when people try to use type information pulled from indexing, a function call, or from an attribute of a class instance. In these cases, we won't know the value, so we cannot perform the requisite checks later on. However: I don't believe that will happen. Most uses are going to be very clear -- one of the major reasons you and other have been pointing out is the documentation aspect of type declarators. If people get funny with their declarators, then it ruins the doc aspect. Even if it does happen, then a simply warning is all that is required. And hey... maybe the *are* doing something complex at runtime with the objects and we should not disallow that construct. > > The typedef unary operator allows a Python programmer to manipulate type > > declarator objects. That will be important for things such as an IDE, a > > debugger, or some more sophisticated analysis tools. > > This is a completely orthogonal issue. There is no syntax in Python for > a traceback or frame object but IDEs can work with traceback and frame > objects. Classes are not created by a unary operator and assignment but > they are still runtime-available objects. The operator is needed to distinguish things like: a = Int or String b = typedef Int or String Both of the above are valid Python statements, but they have entirely different semantics. My point is that I think we want a typedef operator because we want to expression the notion of creating a typedecl object and assigning that to a variable. We don't generate tracebacks and frames out of the blue -- the interpreter gives us those in particular cases. Type declarators are quite different. Classes and functions are different: they have an associated suite, so an assignment does not make sense for them. A typedecl can be used within an expression or assigned to an object. Using the "typedef" unary operator makes more sense, than to have a side-effect from a "decl typedef" statement. The use of an operator also avoids the artificial distinction that you are trying to draw between compile-time and run-time syntax. Those two are so intertwined that a distinction ought to be avoided. Cheers, -g -- Greg Stein, http://www.lyra.org/ From skaller@maxtal.com.au Mon Jan 3 00:32:58 2000 From: skaller@maxtal.com.au (skaller) Date: Mon, 03 Jan 2000 11:32:58 +1100 Subject: Anti-poking lobby (was:Re: [Types-sig] type declaration syntax) References: <20000102190145574.AAA49.241@max41122.izone.net.au> Message-ID: <386FEE3A.2946BC2A@maxtal.com.au> John Machin wrote: > > > previous_stdout = sys.divert_stdout(new_stdout_file-like_object) > > > > Of course, but the issue is compatibility with > > existing code. > > Can't we overcome such issues with much better alternatives, sniffer > kits which find instances of the soon-to-be-forbidden constructs, a > little marketing, and a positive attitude? Define 'better'. For Viper, I plan to provide bimodal behaviour (action varies depending on whether the file is a .py file or a .vy file) together with various methods of controlling processing. Certain features, such as module.attr = value # assign to module attr 0777 # octal or decimal? sys.argv[2] = "New value" ide?tifier # any utf8 encoded ISO-10646 letter allowed are deprecated or the semantics varies from CPython 1.5.2. Behaviours include warnings (all, one per file, none), compilation error, run time error, etc... Tools which interconvert are also contructible. This provides some control over certain features. Maintaining compatibility is part of a professional language development strategy. My point is: even if all the things you mention above are provided -- education, sniffer kits, marketing, etc - there still needs to be a mechanism to support existing code, if possible. :-( -- John (Max) Skaller, mailto:skaller@maxtal.com.au 10/1 Toxteth Rd Glebe NSW 2037 Australia homepage: http://www.maxtal.com.au/~skaller voice: 61-2-9660-0850 From gstein@lyra.org Mon Jan 3 02:07:09 2000 From: gstein@lyra.org (Greg Stein) Date: Sun, 2 Jan 2000 18:07:09 -0800 (PST) Subject: [Types-sig] feedback: PyDL RFC 0.4 In-Reply-To: <386B9A07.57234970@prescod.net> Message-ID: Time for my swing at this... :-) On Thu, 30 Dec 1999, Paul Prescod wrote: >... > Interfaces are either complete or incomplete. An incomplete interface > takes parameters and a complete interface does not. It is not possible > to create Python objects that conform to incomplete interfaces. They > are just a reuse mechanism analogous to functions in Python. An > example of an incomplete interface would be "Sequence". It is > incomplete because we need to define the interface of the contents of > the sequence. Wouldn't these be called "abstract interfaces" or "parameterized intefaces"? That seems to be a more standard terminology. > In an interface expression the programmer can provide parameters to > generate a new interface. Maybe abstract vs concrete interfaces? > Typedefs allow us to give names to complete or incomplete interfaces > described by interface expressions. Typedefs are an interface > expression re-use mechanism. typedefs are also used to assign names to things like "Int or String". I don't see "Int" as an interface (even though it probably is in *theory*, it doesn't seem that way in layman usage). >... > The Python compiler invokes the static interface interpreter and > optionally the interface checker on a Python file and its associated > PyDL file. Typically a PyDL file is associated with a Python file > through placement in the same path with the same base name and a > ".pydl" or ".gpydl" extension. If both are avaiable, the module's > interface is created by combining the declarations in the ".pydl" and > ".gpydl" files. These were to be '.pyi' or '.pi' files. And I still don't understand the need to specify that *two* files exist. Why are we going to look for two files? Isn't one sufficient? >... > Once it interprets the Python code, the interface objects are > available to the runtime code through a special namespace called the > "interface namespace". There is one such namespace per module. It is > accessible from the module's namespace via the name "__interfaces__". I think interfaces have names just like any other object. The interfaces are found in whatever context the definition appeared in: a module, a class, or a local namespace. ---- a.py ---- interface foo1: ... class Bar: interface foo2: ... ... def Baz(): interface foo3: ... ... -------------- In the above example, we have three interface objects. One is available via the name "foo1" in the module-level namespace. One is available as "Bar.foo2" (via the class' namespace, and the class is in the module namespace). The third, foo2, is only available within the function Baz(). I do not believe there is a need to place the interfaces into a distinct namespace. I'd be happy to hear one (besides forward-refs, which can be handled by an incomplete interface definition). > This namespace is interposed in the name search order between the > module's namespace and the built-in namespace. Again: I don't think we want to search this namespace. > Built-in Interfaces: > ==================== > > Any > Number > Integral > Int > Long > Float > Complex > Sequence > String > Record > Mapping > Modules > Callable > Class > Function > Methods > UnboundMethods > BoundMethods > Null > File What does "builtin" mean? That these interfaces are magically predefined somewhere and available anywhere? Note: you probably want to remove the plural from "Modules", "Methods", and *Methods. What is the "Null" interface? Is that supposed to be None's interface? I don't believe that we need a name for None's interface, do we? And why introduce a name like "Null"? That doesn't seem very descriptive of the interface; something like NoneInterface might be better. > Certain interfaces may have only one implementation. These "primitive" > types > are Int, Long, Float, String, UnboundMethods, BoundMethods, Module, > Function > and Null. Over time this list may get shorter as the Python > implementation is generalized to work mostly by interfaces. I don't understand what you're saying here. This paragraph doesn't seem to be relevant. > Note: In rare cases it may be necessary to create new primitive > types with only a single implementation (such as "window handle" or > "file handle"). This is the case when the object's actual bit-pattern > is more important than its interface. Huh? > Note: The Python interface graph may not always be a tree. For > instance there might someday be a type that is both a mapping and a > sequence. In the above statement, you're mixing up implementations (which can use disjoint interfaces) with the interface hierarchy. Or by "type" are you referring to a new interface which combines a couple interfaces? Note that I think it is quite valid to state that interfaces must always be a tree, although I don't see any reason to avoid multiple-inheritance. >... > Interface expression language: > ============================== These are normally called "type declarators". I would suggest using standard terminology here. >... > 1. refer to an interface by name. The name can either be simple or > it may be of the form "module.interfacename" where "interfacename" > is a name in one of two PyDL files for the named module. Just use the "dotted_name" construct here -- that is well-defined by the Python grammar already. It also provides for things like "os.path.SomeInterface". Note that interfaces do *not* have to occur in a PyDL module. Leave the spec open for a combined syntax -- we shouldn't be required to declare all interfaces in separate files. >... > 2. make a union of two or more interfaces: >... > Two union expressions X and Y are equivalent if their lists are the > same length and each element in X has an equivalent in Y and vice > versa. IntOrString = typedef Int or String IntOrStringOrTuple = typedef Int Or String or Tuple IST2 = typedef IntOrString or Tuple assert IntOrStringOrTuple == IST2 In other words, the lengths do not have to be equal. A precondition is that all union typedecls must be "flattened" to remove other unions. The resulting, flattened list must then follow your equivalency algorithm. > 3. parameterize a interface: > > Array( Int, 50 ) > Array( length=50, elements=Int ) > > Note that the arguments can be either interface expressions or simple > Python expressions. A "simple" Python expression is an expression that > does not involve a function call or variable reference. I disagree with the notion of expressions for the parameter values. I think our only form of parameterization is with typedecl objects. The type checker is only going to be dealing with type information -- expression values as part of an interface don't make sense at compile time. I think each parameter will be a type declarator. >... > 4. use a syntactic shortcut: > > [Foo] => Sequence( Foo ) # sequence of Foo's > {String:Int} => Mapping( String, Int ) # Mapping from A's to B's > (A,B,C) => Record( A, B, C ) # 3-element sequence of interface a, > followed > # by b followed by c Case is significant; your example and comment do no match. >... > 5. generate a callable interface: > > def( Arg1 as Type1, Arg2 as Type2 ) -> ReturnType Colons, please. "as" has the wrong semantic. > The argument name may be elided: > > def( Int, String ) -> None > > Note: this is provided for compatibiity with libraries and tools that > may not support named arguments. I agree. The return type should also be optional. Note that we can't allow just a name (and no type), as that would be ambiguous with just a type name. >... > It is possible to declare variable length argument lists. They must > always be declared as sequences but the element interface may vary. > > def( Arg1 as String, * as [Int] ) -> Int > # callable taking a String, and some un-named Int > # arguments > > Finally, it is possible to declare keyword argument lists. They must > always be declared as mappings from string to some interface. I think that I agree with the "string to " argument, but it is interesting to note: >>> def foo(**kw): ... print kw ... >>> apply(foo,(),{0:1}) {0: 1} >>> :-) >... > Note that at this point in time, every Python callable returns > something, even if it is None. The return value can be named, > merely as documentation: > > def( Arg1 as Int , ** as {String: Int}) - > ReturnCode as Int Ack! ... no, I do not think we should allow names in there. Return values are never named and would never be used. Parameters actually have names, which the values are bound to. A return value name also introduces a minor problem in the grammar (is the name a name for the return value or a type name?). >... > 2. Basic attribute interface declarations: > > decl myint as Int # basic > decl intarr as Array( Int, 50 ) # parameterized > decl intarr2 as Array( size = 40, elements = Int ) # using keyword > syntax "as" does make sense in this context, but I'd use colons for consistency. > Attribute declarations are not parameteriable. Furthermore, they must > resolve to complete interfaces. Agreed. > So this is allowed: > > class (_X,_Y) spam( A, B ): > decl someInstanceMember as _X > decl someOtherMember as Array( _X, 50 ) > > .... You haven't introduced this syntax before. Is this a class definition? I presume this is also intended to create a parameterizable typedecl object for the class? e.g. where we could do: def somefunc(x: spam(Int,String)): ... > These are NOT allowed: > > decl someModuleMember(_X) as Array( _X, 50 ) Reason: modules are not parameterizable. However: I think modules should be able to conform to an interface. And since an interface can be parameterized, then this means that a module can be parameterized. This is analogous to parameterizing a class. > class (_Y) spam( A, B ): > decl someInstanceMember(_X) as Array( _X, 50 ) > > Because that would allow you to create a "spam" without getting around > to saying what _X is for that spam's someInstanceMember. That would > disallow static type checking. Agreed. The _X must occur in the class declaration statement. >... > It is possible to allow _X to vary to some extent but still require it > to always be a Number: > > decl Add(_X as Number) as def( a as _X, b as _X )-> _X Note that this implies the concept of hierarchy among the interfaces. Either that, or you need to define a way to show that _X implies the Number interface because it has the same members and each member conforms to the equivalent Number member. Note that you will then have to define a rule for whether "decl x as Int" is the "same" as "decl x as Number". For conformance, is the first too specific, or is it just a more concrete form of the latter? (but still allowed) >... > 4. Class Declarations > > A class is a callable object that can be subclassed. Currently the > only way to make those (short of magic) is with a class declaration, Proper terminology is "... with a class definition, ..." > but one could imagine that there might someday be an __subclass__ > magic method that would allow any old object instance to also stand in > as a class. eh? what is this doing in here? > The syntax for a class definition is identical to that for a function > with the keyword "def" replaced by "class". What we are really > defining is the constructor. The signature of the created object can > be described in an interface declaration. Ick. We don't need anything special for this. The constructor is given by the __init__ that occurs in the interface. > decl TreeNode(_X) as class( > a as _X, > Right as TreeNode( _X ) or None, > Left as TreeNode( _X ) or None ) > -> ParentClasses, Interfaces This would be: class (_X) TreeNode(ParentClasses): __interfaces__ = Interfaces def __init__(self, a: _X, Right: TreeNode(_X) or None, Left: TreeNode(_X) or None): ... If you're just trying to create the notion of a factory, then "def" is appropriate: decl TreeNode(_X): def(a: _X, Right: TreeNode(_X) or None, Left: TreeNode(_X) or None) \ -> (ParentClasses or Interfaces) IntTree = typedef TreeNode(Int) Note that parens are needed on the return type so that the "or" binds properly. >... > 6. Typedefs: > > Typedefs allow interfaces to be renamed and for parameterized > variations of interfaces to be given names. > > typedef PositiveInt as BoundedInt( 0, maxint ) > typedef NegativeInt as BoundedInt( max=-1, min=minint ) > typedef NullableInt as Int or None > typedef Dictionary(_Y) as {String:_Y} These should be assignments and use a unary operator. The operator is much more flexible: print_typedecl_object(typedef Int or String) Can't do that with a typedef or decl *statement*. Also note that your BoundedInt example is a *runtime* parameterization. The type checker can't do anything about: decl x: PositiveInt x = -1 But we *can* check something like this: def foo(x: NegativeInt): ... decl y: PositiveInt y = 5 foo(y) But this latter case is more along the lines of naming a particular type of Int. The syntax could very well be something like: decl PositiveInt: subtype Int decl NegativeInt: subtype Int The type-checker would know that PositiveInt is related somehow to Int (and it would have to issue warnings when mixed). It would also view PositiveInt and NegativeInt as different (thereby creating the capability for the warning in the foo(y) example above). Anyhow... as I mentioned above, we should only be allowing typedecl parameters. We can't type-check value-based parameters. If you want to introduce a type name for a runtime type-enforcement (a valid concept! such as your PositiveInt thing), then we should allow full expressions and other kinds of fun in the parameter expressions (since the runtime type should be createable with anything it wants; we've already given up all hope on it). But then we get into problems trying to distinguish between a type declarator and an expression. For example: MyType = typedef ParamType(0, Int or String) In this example, the first is an expression, but the second should be a type declarator. Figuring out which is which is tricky for the parser. As a consequence: I would recommend *only* allowing for type declarators and skipping the notion of type-checking with runtime types. Introducing the "subtype" thing that I blue-skied is possible, but I'd punt on that, too. It seems a bit too specialized and probably applicable with just a few types. > New Module Syntax: > ====================== > In a future version of Python, declarations will be allowed in Python > code and will have the same meanings. They will be extracted to a > generated PyDL file and evaluated there (along with hand-written > declarations in the PyDL file). I disagree that they will always be extracted into a separate PyDL file. As an optimization: sure, we could do this. Effectively like caching a module's bytecodes in a .pyc file. But I don't think you should codify that here. >... > "typesafe": > =========== > In addition to decl and typedecl the keyword "typesafe" can be used to > indicate that a function or method uses types in such a way that each > operation can be checked at compile time and demonstrated not to call > any function or operation with the wrong types. What about the problem of non-existence? How "safe" is "typesafe"? And how is this different from regular type checking? >... > An interface checker's job is to ensure that methods that claim to be > typesafe actually are. It must report and refuse to compile modules > that misuse the keyword and may not refuse to compile modules that do > not. That last sentence is awkward. Can you rephrase/split/etc? > The interface checker may optionally warn the programmer about > other suspect constructs in Python code. > > Note: typesafe is the only change to class definitions or module > definitions syntax. Class definitions also have the parameterization syntax change: class (_X) Foo(Super): decl node: _X ... Class and modules should also have a syntax for specifying the interface(s) they conform to. I don't think this requires a syntax change, though, as I would recommend assigning the interfaces to an __interfaces__ attribute. [ note: JimF's Scarecrow proposal mentions __interfaces__ but seems to actually use __implements__ ] > "as" > ==== > The "as" operator takes an expression and an interface expression and > verifies at runtime that the expression evaluates to an object that > conforms to the interface described by the expression. "as" is the wrong semantic (it implies you want to use/coerce the value to a specific type, which is impossible). Use "!" or "isa". > > It returns the expression's value if it succeeds and raises > TypeAssertionError (a subtype of AssertionError) otherwise. > > foostr = foo as [String] # verifies that foo is a string and > # re-assigns it. Don't you mean "list of string", or should you drop the brackets? >... > Interface objects > ================= > > Every interface object (remember, interfaces are just Python objects!) > has the following method : > > __conforms__ : def (obj: Any ) -> boolean Just call it "conforms". There is no need to "hide" this method since the interface does not expose interface members as its *own* members. > This method can be used at runtime to determine whether an object > conforms to the interface. It would check the signature for sure but > might also check the actual values of particular attributes. I think that you would want a version that just checks an objects __interfaces__ attribute (quick), and a different method that does an exhaustive check of the object's apparent interface against the specified interface. >... > Experimental syntax: > ==================== > > There is a backwards compatible syntax for embedding declarations in a > Python 1.5x file: > > "decl","myint as Integer" Just use a single string. The parse tree actually gets even uglier if you put that comma in there :-). We can pull the "decl" out just as easily if it is the first part of a "decl myint: Integer". > "typedef","PositiveInteger as BoundedInt( 0, maxint )" Since I think this would be a unary operator, I'd recommend a transitional syntax of: typedef("TreeNode(Int)") i.e. we parse the string, but it also calls a runtime function to construct the typedecl object. The issue here is that we want to inject certain names into the namespace. "typedef" as a "statement string" will be ignored by the interpreter. As a function with a result, which then gets assigned, we get the right semantic in both compile- and run- time cases. > "typesafe" > def ...( ... ): ... > > "typesafe module" No problem. We do have an issue with adding parameters to a class definition. > There will be a tool that extracts these declarations from a Python > file to generate a .gpydl (for "generated PyDL") file. Why pull them out? Leave them in the file and use them there. No need to replicate the stuff to somewhere else (and then try to deal with the resulting synchronization issues). >... > The "as" keyword is replaced in the backwards-compatible syntax with You didn't put anything here :-) I'd say the temporary syntax could be another function call: assert_type(x, "Int or String") This would work properly at compile- and run- time. > Summary of Major Runtime Implications: > ====================================== > All of the named interfaces defined in a PyDL file are available in > the "__interfaces__" dictionary that is searched between the module > dictionary and the built-in dictionary. __interfaces__ for the module should specify the module's interface conformace. The actual interface objects would just be other names in the module's namespace. > The runtime should not allow an assignment or function call to violate > the declarations in the PyDL file. In an "optimized speed mode" those > checks would be disabled. In non-optimized mode, these assignments > would generate an IncompatibleAssignmentError. This is a difficult requirement for the runtime. I would suggest moving this to a V2 requirement. > The runtime should not allow a read from an unassigned attribute. It > should raise NotAssignedError if it detects this at runtime instead of > at compile time. Huh? We already have a definition for this. It raises NameError or AttributeError. Please don't redefine this behavior. >... > Idea: The Undefined Object: > =========================== You haven't addressed any of my concerns with this object. Even though you've listed it under the "future" section, I think you're still going to have some serious [implementation] problems with this concept. Cheers, -g -- Greg Stein, http://www.lyra.org/ From Tony Lownds Mon Jan 3 04:47:47 2000 From: Tony Lownds (Tony Lownds) Date: Sun, 2 Jan 2000 20:47:47 -0800 (PST) Subject: [Types-sig] feedback: PyDL RFC 0.4 In-Reply-To: Message-ID: On Sun, 2 Jan 2000, Greg Stein wrote: > On Thu, 30 Dec 1999, Paul Prescod wrote: > >... > > Interfaces are either complete or incomplete. An incomplete interface > > takes parameters and a complete interface does not. It is not possible > > to create Python objects that conform to incomplete interfaces. They > > are just a reuse mechanism analogous to functions in Python. An > > example of an incomplete interface would be "Sequence". It is > > incomplete because we need to define the interface of the contents of > > the sequence. Why not allow a default type to a parameterized interface? class (T:=Any) Set: ... decl f as def(x: Set) decl g as def(x: Set(Int)) > What is the "Null" interface? Is that supposed to be None's interface? I > don't believe that we need a name for None's interface, do we? And why > introduce a name like "Null"? That doesn't seem very descriptive of the > interface; something like NoneInterface might be better. > You need to be able to specify that a function returns None. Just using None is better than introducing another name IMO def f() -> None: ... > > 4. use a syntactic shortcut: > > > > [Foo] => Sequence( Foo ) # sequence of Foo's > > {String:Int} => Mapping( String, Int ) # Mapping from A's to B's > > (A,B,C) => Record( A, B, C ) # 3-element sequence of interface a, > > followed > > # by b followed by c Paul, is it your intention that... a tuple is a Sequence? a list can be a Record? I hope so, I think these syntactic shortcuts will be good if so. Can one declare a Mapping of more than one pair of types? ie, {T1: T2, T3: T4} I think this is different than: {T1: T2} or {T3: T4}, so it should be possible to declare a Mapping of more than one pair of types. decl x as {Int: String, String: Int} decl y as {Int: String} or {String: Int} x = {1: "one", "one": 1} # should be allowed. y = {1: "one", "one": 1} # should not be allowed. y has been # assigned a value that is neither {Int: String} nor {String: Int} Also, the type of x.items() would be: [(String or Int, Int or String)] but the type of y.items() would be: [(Int, String)] or [(String, Int)] > >... > > 5. generate a callable interface: > > > > def( Arg1 as Type1, Arg2 as Type2 ) -> ReturnType > > Colons, please. "as" has the wrong semantic. > I vote for colons too. > > The argument name may be elided: > > > > def( Int, String ) -> None > > > > Note: this is provided for compatibiity with libraries and tools that > > may not support named arguments. > > I agree. The return type should also be optional. Note that we can't allow > just a name (and no type), as that would be ambiguous with just a type > name. > The return type if not specified is... Any? or None? > >... > > It is possible to declare variable length argument lists. They must > > always be declared as sequences but the element interface may vary. > > > > def( Arg1 as String, * as [Int] ) -> Int > > # callable taking a String, and some un-named Int > > # arguments > > > > Finally, it is possible to declare keyword argument lists. They must > > always be declared as mappings from string to some interface. You dont show a way to specify default arguments. How about: def(Arg1: String, Arg2: String = optional) -> Int That is, the syntax is just like the normal declaration syntax but the word "optional" is used in place of any actual value. For truly in-line type declarations (ie, the same line as the function definitions) using optional is not necessary. > >... > > 4. Class Declarations ... > > Note: typesafe is the only change to class definitions or module > > definitions syntax. > > Class definitions also have the parameterization syntax change: > > class (_X) Foo(Super): > decl node: _X > ... > > Class and modules should also have a syntax for specifying the > interface(s) they conform to. I don't think this requires a syntax change, > though, as I would recommend assigning the interfaces to an __interfaces__ > attribute. > Hmmm, I dont like that. Seems too magical for the type checker to look for assignments to that. I'd prefer this: class (_X) UserList -> [_X]: ... What about declaring a subclass of a parameterized class. class (_X) SortedList(UserList(_X)): ... class (_X, _Y) ListDict(UserDict(_X, [_Y])): ... This seemed like a good syntax at first glance to me but it is ambiguous because base specifications can be an expression in Python. So, how about: # the superclass class (_X) A: ... # parameterized subclass class (_X) B(A) -> A(_X): ... # concrete subclass class C(A) -> A(Int): ... -Tony Lownds From gstein@lyra.org Mon Jan 3 13:29:18 2000 From: gstein@lyra.org (Greg Stein) Date: Mon, 3 Jan 2000 05:29:18 -0800 (PST) Subject: [Types-sig] Happy New Year! ... and a new demo/prototype Message-ID: This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. Send mail to mime@docserver.cac.washington.edu for more info. --1658348780-1058415798-946906158=:412 Content-Type: TEXT/PLAIN; charset=US-ASCII Hey gang! I've attached a new round of work. This one is *much* improved over the last version. Hopefully, this will help to provide a little more reassurance that this "works" and to provide some more demos. Specifically, I've added the following features: * an interface file parser(!) * associating modules with an interface that was parsed (this means that "import foo" will create the name "foo" with a TDModule which might have the "correct" interface) * calling a class object now "returns" an instance object * classes now record their implied interface. For example: class Foo: def bar(self): pass baz = 5 f = Foo() # check.py knows "f" is an instance of Foo f.bar # it knows this is a method f.baz # it knows this is an integer f.nope # it issues an error because "nope" is not there [ it cannot track instance attributes created through things like "self.foo = 5" ... these will need declarations ] * calling a function "returns" the functions return type * the first parameter of a method is defined as an instance of the class where the method appears * other miscellaneous cleanup/refinements/etc * whatever I forgot to mention :-) The syntax for the interface files is basically a subset of that discussed on the list here. See iparser.py for the formal syntax and "test.pi" for some examples. Note that the only dotted_names that it understands are Int, String, Any, and None. various.pi provides some interfaces for a few modules. This makes checking of check.py a lot quieter. You can invoke it like so: % ./check.py check.py various.pi As always, have fun! Cheers, -g p.s. note that in the example above, I did not have to declare "f" -- the checker is well-aware of its type -- Greg Stein, http://www.lyra.org/ --1658348780-1058415798-946906158=:412 Content-Type: APPLICATION/octet-stream; name="typesys.tar.gz" Content-Transfer-Encoding: BASE64 Content-ID: Content-Description: Content-Disposition: attachment; filename="typesys.tar.gz" H4sIAEydcDgAA+19+1cbR9Lo/hr+igkcX0m2kBG2k+9qg3OwwRvO2uAFvI9L uPoGaYBZSxqtZmTMxrl/+61HP6p7uiXZsZ39zmFOYjTTr+rq6qrq6urqwXU2 eNuZ3v7hCz5b3a2t7588+cMWPN9/x3+7jx/TX/jSffL40R+2vt969LjbfbL9 /XeQv/v94yd/SLa+JFD6mZdVOkuSP1yVVZZPFuXLZuXXAOjrPhvfPpyXs4cX +eRhNnmXTG+r62KytrG2kQwUZSSbm/w7qa6zpLqdZsm8TK+yJJ8kafKaCiTj YjgfZVBuLR9Pi1mVTNMZIEy/lbfji2Kk36ribTaxSaX5Wc3yyZXJBS2Va/Jt mA1MHaN36Wie6bdJOs7KaTowH3LV/hp0JJ0k2ftBNq1yALQqAPosuSxmybt0 lhfzEpqdX16ubWxsJJf5e+hkXibzaafTWctmM8i2k/T7WH+/nzxIGh362ICK B6O0LJPniJls1ltLkmF2CVnzSV71+80yG122k3fZ7KIos52tFmZIEvzaUR+h YvVrzab1hzn0oxpcQ+ovya+UwLBCb9vUT8Q747MDfwi0Tl5l47KpGkmS/DK5 Tsu0qmYKjka/AcBjVpPHb/CMmjiHdm1mCVg+gYkyGWT9atgfpEAQCkLVcaKQ /mU+ylST+FM1BuDg8DXpU7Kzw0PbOaHhPoXfGibMANUW02xCmQHwWaNlgeBG qux9RcmdWZYOm62WCwMlMwxlMZ8NNBRpWUHdTBedcg4oa6p0Sq5mWWbTIfN2 NZ9Cb+BXO+kKIPqqGcjfxH+o+Q2YJb/lwSqomtfHB3/dPd1PXu2f/nS0d0Lf NW2Jhrl/1LzuHYBd0ZezrXNEsqIRRBQM3nReidG8nE8GJfT2LDn/Bsge3nBy lDg7eKojjWWDYjIEhJQll9yATxVMDZj1RIezrCxGQMIwwYbwbTCfzbKJmIud 5CZLbvLRiGZcqup4M4GWDnWe5CIbpJhM0w7+Qx6zfjUqLtLRuq0qubnOYU5M smxYqnoA1Fk2zsYXAMB8QtAXl1yioQjM9teAuiPAO1Yfm2JwueX+BJEjf9tS LvyyrG6kM52X101TuqWxBzMoIy4H+EpnQ8UzuWrKYop0kNOls6zZ0Kyn0TYc UM2bVrzIsBhgCTPR3cfUQ1jr69ey2fRaiFUQf0wF1fCwmGSt1gIgL+b5qAKU uH17M3k7KW4mrwg1CnPI/ybFkBgf0Xe3dy44HSYhyX+7w4Kls3+492r3+M/7 x0SYteTD/b+9PDjct3xQTR2d0U6dshpXHrfET03KCUCYkYWJ8o5mnxKQRIr4 wj2neQWvuaLUxhA5f4k06+EHZkKfsjRN3dNZMchA0GTAc3W1/5pn82xo5y1n VQUmRZX15DQfp7fJ1ay4SYZzHFaeaqOimCL2UlML8hgUXvBT1cRPMYCZXSL1 5jAGKVR/DTRuRgaLM7vQDWrMehPHzhmkjXYiJojAsDuJzOxBcBTxJE3AZjZJ L2DuwNDlV5Mxsh1STLIJgDTI8EMrSYHzILZ0B01FwOMBlgqGoKMFD5IYfANK HGJnsAgQcSedXYG8Ky2tCBrW+Q2EUjpgBU2qBYkFqindbLabxbT5eSXIySnI j1f7h6fJ4dHevitBiHxZdhi4epq5QxrQDShQIDDwJfkAhA3q1Hwy7JupIDuJ VTR1PTB1VD+cQj0gMl0bMHFbNaBc/6xmt7oBTXTm+UBjgcADNKhxwU/I13ca QR0N/4Is0+oMTA8c+xK1I1DlqOpyDCShoMreT037U5gWFb/A2zAbmQRoz8A7 Km7CUH5IWOnUORVhq7fsfTbQv5nT6Fr6Fh7TgSU9MED0kgtQft7acZoAL50b 5M6yaj6bmLc0LzPTqqljtUaZbixVRMnHZukJTCfNxh8b4r11PzmDL+eJYsOa zWWofqZVxnwOZGRFsxh0uLf5dMpsC+DKxvmgGKGwR96u6/ClhAZOz1uPw0OO pmbxBrC2Eg8n+68O2q6o0AxKCBshJCxR+bLCpLgSgzCqZkUMmyq5lzTyS1Bn MtBdGz3AI6qtgNJsVP9MmM1GZdaw384VemFtg7oTyqFxOsHFG0kZkida/lth myOGZunkKmtutZMRKOMGvmQzedROHptVRJwd5LCG6J63kLgQzpXyP+L8BPqa Xjq47d9LHiPyHy0HYBNbF5UR2i0LimHe5gDk00sN/78NzZGcrENcZbDECgxL vJvcS4PieMZHAh1B1D5NHi/H6ncBpGpWHkOpTgeEws8G8d5RjijNJ4xcfvs9 Ecz2BNRYZN9Zv/CRreHngvDjdM8rF8K87qftyjXqZAWukgCw/PKWWJyqj9QZ 5HGwmp2lg4qsGKNsfLr3I5Xn38mOVZ13J7dmzkg7gx7T0TsYBFwBcVc7MBTj tOqrjjfVX83rSCYmjVE+yZJ7wx6OYS+5h/i5B+ure4mU+2fb521VfVsB1rK9 5AXlzXWGeqPu3piEDS/3uGVcXJoy6XQ6qqGjCTTKirQeLdvIv+Y5NHKdopWq QM0PUMsSSeiIpUZPXmprhu52W2Pl5V/xz6G0lLgKmyIJVa7DWqDT6TghPJH8 LTIH/2v5HPwvj03SJNRKVGwS6nSYhPCzIcUJW8f6oF7hStwKlAfOOgCf+sQE 9cKvrwHLCBgB+U0LedkQAMLvjeSMGOxZo83c4Dww3WGelMVY2R95RuOQfmsW 17BCgUmC3XzI1SYgyfhdwfOjRrmYmEKQu8CZxYKtcOm4bNM8H6e4GjK9ronU R55IbSePhFGOVJgdC2J+bpKC6kcQao+0kKy2ZVqkE9gBJ5dXx2O3jhWY5tnj c7PuEZWG2JOs9WMYlX18lqXIQFMbMS9gkoZ7wfALzlWDE6nv5jqtFItGu0SS XhRzXuaT0VjxFqZQ+Aqzh4H7MVAXK6g37eSfcyB2nGmGY7sZP4KXOVj9OK7m 0cJi/iaFjET8AnXuwbaSlWIQnGmBMswd/rohRqnhu6/2sSrIripaHYQ1v6EN yRJWmdJL8zwRar1arEY1IU4G1gf/NhLsmLBG1PilTYISTeCT7+AD1IdC8Txp tBoqn/jcS5qXUwTlrLGjuGkCfLV1v8bL6Wk27iswiPvC6/3GB/wG/7coATl8 4HMrVB+skv22m1it/5WWKu3GeVDHR5w7Kv4GofayKOwGB2gVsPwFWeOYr9Au k1Ywzar0La4YJzB7ZldzOV2qoVSZTvdeqNJNSeFttIu3EzZTkSGzRkUwO3La TxBQw3/uepMzufxaDJVdVaqMsGKp0b2dH84wJzd5dQ29V59Q1r3Nbm+K2XCT 3olybFGxt2U+uqiAbr9w7WTu7HRbLyagnqHRAWQdYnqWXc0B+dxsaZkaWf94 n8F8kwt11XFh0KVSCq+ADFegTeS+BtKUy8yotQ7ojtlk2OT5Srns8lvyLlxD OzUiaboVkggAIptlk0YFCM6MIQLY5XxUJbP86rpirn4BguH+sLivN1AqpyaG BlvwrcdmfLR4uBylVZVNjPmScUh7LMSaUWwoeTN2Ed3Hoh62aQXLyQ6KcQNH WWdraCZlmzYLF23W+Q1rxFNBJ88wG4Va8tm/3cQKtP8S1wHwq9742Vave676 YlKV4TZEemoL1QLuiCAup/vSDIg/OW6ELTSnGy4Evy8ypBKypoNmgBOVdsrJ etnGYZuwWTqflZWoSrMq3jK7wNVfogU6G/7TiisxheitTxPZFeIVWpU7Ollq kPqboypCK6KW/lVWye3epi7TcskEmbbCFw3Uljd5zOw+R6xAb2gyduznM+Z3 mMgyCIYHxFzHqQbVOWDkO34xy2358bW//ePjo+NeMs5h0sBwrGPX1q1ERS2Q q3Yp0NJNs4FFGm2NntYS2uWSxMCTWllJM7jzNbTOFFZuRVhzVEqpJl1JZVZO 6E0xqIBPl9OM5GHb2Q1RE6HE/aLkNoO107s0H+GuiqnByX5RDG8hOzLCcj6d guZOWm+ZZYZLwfzFRRjUidZ54DtDpSSg6qcUyz8megMluSnmgIfpLHuHVA/y 3Ci3dhntq6XCEgP/ie2XDd4Uw+1pB2yc8jDHs5neSEMPEyxA+1Y7yV52mc1m 2dCgWLfAdRtAaGdLswV8ETrfdJHGR2KKyfsDKXGXU7Z2Ge2NP/SUlmQVppqW JE0uMT2BLf9O1q6qwRfGIYs54XXx9urzo1evdlfYPaU+uAJqBdms4adNcY1h vfETQ7JOBy2ZfhqVtinNi63GudCwDZXrDVarbJttXa1jTrIbbsNMSta/kNLU jvXBBEpektdDgQwb1Ibr9J1W5DEjV6DEQtv4RgzQqnGBiz0U8fPpEJskmWIm O1UieL3gDM/xq9i596Fpt5ZzBAZsNXbAef8zeYEVUz7WtWWKEc6jqkwXQPt5 BfrzzUQ4kRHBxnavdTNtz2Mk6gkiXEA0rXn0kDNDhnV+E4EBKrwuQAhpRWCo eBQOV1H3RKpve6yFQNEbzQgHSHiCAa0KeYZb3opeFKsZMKFNyg7rAiq1KYhJ max/mJRPk/QqRUHHVZrMoBU00cMATQZ68DV56+6baTQooIeDyoNE47pjPuOo NBnAtmXBZjc3xiFMhp5lCE29HsU3WCB7uCHhrBwusDjKDrU4wtclNn87DBta bzeVKiO29j+0DkDLN9+228m2WQWvtmeR25VG1PT2sSY3X9niNiMbBF1vg4CQ Z/06alsECrsLtgj8TQJVYtEmwSeY1j7BpLbKVoHtPpGudT6I0a7NAfKNXhpJ k7ck0bDD1vPzuq+UHYCPEuhBmrbwaveIGLQ6nWxco4a3WbbqLtsX3MnC1ejC PSwxNtr/Izo0OgOODAoa+q5dJnmj3/qGRDf7bRbc8Ke3RjIsqiobkv8fK4Xi Q9icB/rl5awYu0VNfWTq+8CaEdVHZrz72pAnyvR0po7OFFRBu6Tt6fqND+8C P4zIHoZsWq7FhWYZsIxJ0/kGW6SIwxbTzRGoESOj0WgjPfoDJ1ez9MIuv1Hp k4YabRZRTWrN2TIQ7aaaGiZSzW61J6cVUug7YwtCp6/6IvE6Lftvs9um54+N j6/fsSNksynLn2G587ZnzPLXo6TwXZNaRZb3ArUKXAlfZIDtmQuh37Djhilx EFbEjCecD0qQHT2S1j6XIJQLzunuseyMP4P/tnt8eHD4p16yjuRusM+e//fX obHr/AIVKZK2xGOg4x2c8mqby1Tu401Z/IYFGfxQI9P1rIgEZ6NE8wDhERbj ASIL8AB+a9Qma529GPeyuOajMtB+azZgjgxrI+174ey71uoXLmuxFkQWaIPf alWv4MaieYy/I262Lhd6WBiIrWNcDGCbA+CllwDfdlzqogtPmQlXn+o9UJ9w yYvVJrJAXfymNsdpxyfiEgXzxqBIWPFiWF6mC/qzTa3IIyJTLbe4i8bPMNpD kwM7iC+1zX9JjgF6VAc3Ij6ImOj7sConvuTgcA+dYvHrg2RvH1+i+OwGPAdd y4v2/rNNOUQqHSYdmdkLbEl+gsh0/Q3j3umfzat4/++vj/dPTg6ODoVbMSac 0ra3IpI0QRv9nt2JKy7+Cau6NemDrMk56qii0nmlxtwPf7nmMNLxTvd8VfIi n8A619TpzBhdhEfYHA7Qy7W4Tqw6p8vjLoaGFY8IbU5TEjC8Xid3+T9ahFAW K0HUd2nczd9nw5fZ5JSOGxkgHXwtwlUPe9JXuEL3Ov0KCPuQjNLxBdrhQgqc Q1Eqo9dnxq1Kq1PyRw+FL135XA/oI2O91KYuaMsHQ6FsRz7q/PMrddRpTEQl l0rvodVLoxA+Nsy7Vn0/cx+pjc/USQ1qrJM6HTgu/LRdU+796SwviwlDvkR6 bEaFtBRBZBxKR7my9KSVahe1ZlxwZ8M/4pGUC3TjmirjqtmjsqfNNCKGQiJr aOPiWOdgB/+kiV/6xZTe9FiOsstqYT+3NGV79pjtmueWtcXwTvCiWoUdhiEF uGBMfsBFWeMp/bvDv/nPD+oPJ33Lbzs7DV3HN4b5f2BH2g8K0folL80fSlAF NUqkkxmMrCvwVCaXRaiPKieT9Y7JShyU3L2Jg963eobKGXZrULSkanE9PwIA bQcWgjYzbnBacNQCFfvuro006JgEyFq4jjJ581IgsZ4zgI4uoUMMx1poiqAZ Kx+kF/kor247qFKW1Ww+qHhiwMTviFJm6XNDh4TRGw2JlJalpmQveZT8kLyH /59oiwmTvOEqB5PKmWWY7hhTF9lRe8n7Ytbn+dX40DBvn8QrHUygxwPt9gew 4MDriWICW4MRA12ns7xU4P/fhnn7ncHXYCySVQx+eZ1fVroD/6sh3n/nLlhA ogq6yQGjMMura9WNZuOHH4CUnj5ttMT3/xyGXWeX1pUFeA4ma470cv/F6clP By9Ohe+U3S6hiUozPqNTodi1hwRKR2+O4Cdrm5lafw/iOMBQZGvHB3/6iZv7 fK1FGYMdmSiRmhw9VnhgZB/AwG7CuOL7/8gRff3yzclXGcxXB4dvTr7COOJI xFcUs3EvuUwHuHpros34Q+Mh/H8PRpC//o8cQ9ek+NnG0G/l5e7JT1+FVF7v Hz/XBowvSyw86FH3FkrtJXqaf2j8P0MoeIAW1P3Z6msLyjiRg7plhlQ7a9bn 5IK+u+sKM2YTb859lppOD17u7X9STcq1vUHYasSWP5Qa3YjCRJCpVTFOqlma j7LZfXbb9mbtkmGIzdmuN2dlqBkxNR3dfO/ozbOX++7Mq4bbi2f3g67vGAsK 9QXaItHjAoqbNAdDwa7RBnFfoaOJ5etcpI5nxGFUvkGacsU3Vll0LEL9/sz9 eE4ff4GPw3xQjdO32Qy+/qqdvviBHP8tXZT+2+zQwZ83r57tH8OPk9NjUPkf RGdHYHq8fG2RLkfI20Y7fi3HZplpqml2nBzj0EI7swfWyV+erQKWyBYC66/p DIBCj+Cmu98ShQ6x2zcaLbCzpZA+O959vr8KrE5G5ZgwYxeXt9ntQ3Zc4KAg 2sflOptlPy7o315Orojp7NZz+Fyxt0hwH9XbZ7vP//yXN0enph/L92nqcLOH eKh6puQga0wrOvnfbBl3DzwUq1zGOgPeUp1CU+wHgmCoat7CRKCi7pZ0DDpy ujV2tObbIB54pvU+voPCoiG2kIUDps+hzO6h+uF4zRO5VYl00l55C5maqO8W R80WEbfp+eQCo2sk7A4Q2fvxPf1rJ0o8RCpOTfs+CcVRayeNOW818wYIFm4Y TqzNzhFmrC3WAD3+Smvno3rMWu069xItrvnEYfyhAzBmQTu/KAezfLpwv8LJ 1LOvvHNhXv8jty/wIKOBF11OaT+DcobN0cu3K0x9S/EFA9dpmP8/JMoofaYO rfUa+ucZDOggK6bnugZ+7Yk8vCJBlyCppXyEQrl3ZBbsyuI4IlNkhBec7j3j +FVNNpfsj0b5tMxL3P6yfn8JepuPQLNDGJXBX9Wk3QqoNrK7kA4osugxMuwH e83njNB3OMWYR0U4MuNNXuXkb6jVS+O+qCIeLTj5tIKCaFjSyNkrlRNdZYGq ums+28GvW4wjf3vV0yCfH708OjRzIHmagHoYVDwfdL3dLPcs10r6pqxw069P k5xm8oV7Shvzw//fIFMquT96r0eVdKH+aGCFse4mo4hemgCM23r+78w2GJy8 REGfdVf4DSx6Dk4P9kNxCgl2ywPU/PfP9rGvc19JTe67DkaEzsfNScuOTiWz C0Lww2Ea6ShEoxJ0Sr4JGJNBOsGqr4GjwlzlupL1e+V60rw3bKHk+1kIOXya XoNnE5SGemLQwpAJGP797RBRVUsAcpvz4Qm4dxg570lfEPZDcpTS7GB2K+Y3 Se6GCoGqaqcD3mKF107egaZ6gaN5W2J8AlBGLotmzXlYvQ2A92EsO/TkMloI Kxz1Xl50qov+JRJ857KPhybgD5/iwiYDiQPoV2dQ9KWqoqYFHpfSnNpEXXWU TJURqFCEUhUU2ArEdXI8WmvUv5EcHe4nRy/0W2LcY/WGJeoMbPtW6oLOuNQt gjKq7O/y1AWlT5K9HWx+wX6ObHw1P4MaFCbuyCdUFuQYnuwywYTQicZjqoQv Ia7C8TACKyiFtDL71xzjBSkWzNkF7zcOohvJvwug3yFOJZLFGEx2PqnyEbLr yxx0NeuASbo5QIc+iRTuTFVB38fZMEcTHbWi/DcvSJgA9FfV9c7Odo81cVrM tFGd2iTwOqqacVFWgbquYGKM01lJq1P0mM7wQHFP87tmMVU/W/f/aNRDUAln yreSzFeTAoPzkmtP2SEb0RyXqdRCiWCxa0FbG53asiZRCZfSVTHk6qRyYWK0 GdJFDWuFYSTmdHI7qdL3+7y4WAdG2qjM+QsL/PrCsYQhoxgdFHoO0QRK14Sj b1JHKbCtHZsU484o451lHtFzcWwT9bP4CgdamjQ8OWsSVJBBBnGizXz6SDGq BQi0OGI0XluEMWDDYxwFLXXJ7qVhMX5qbAkEJUc7p51t90DTcVamkLrU8Pdp 41OHMmTZU3noQOwCs4gmJ78U9Uqe16HBT/XBkAYihge2A5PrHUzGCc10PeF+ V6NggIodptmNrHcCFkLft8KzDq4wfM3W+jL0u4GGXJiO6+bBGkyuaXAFmM7O l8Lk8vkIcIFDtoGaUC3Qk6glzR2LgRzlwK7T0XpQjbDgfVV9gpfU5sCOPDUs 2YF7uEixgiWHhS3DEEI84YrrSyNTWh+kck4RO8SFDMA5RyzOgNXxSoNVw6lT Uhzzcoa1XplmK1yf3mwQ4dcoWfEJgNNnE44ZSXMKdQDn46f0KkRnzsijCr4e qlBMtgWTxzGS6Y7TLHKknGNrqZ0jJVPwDS19BIi8dwbKd34xr8yS+ip/l9ll rVgYj0adToePAqHbGR/BTpqmgpYMGC1iRa9FB34XFX/Rp/j4h2yFNSr4WFuh V+qzWswoCaruxy0+hg+Kc5Bcoh0whbXIJmkWbZyxQ2YpG0YGAZFRZCIkwFcC 6KEm8FHsEj0Yn/LYoFEjNGoMQ/3UcrwpUNWz96GmQg0dYGa/IUEaFAU8aAZd f87LXhWSne9a2JxSiIFCQyljb4Di3FnXCsrKYcR4jKPWr8eBYK7SEB0EfeF5 CYWls8UWxKRuBZfiZcERaOkrokuH5EHNZ8SRCrqk1+naLmFA3q60DI/q/m4M WDdA0RcRrHb/QzUYRRaLTgk+Ip5i+zxNYBgaeDbbW/aMspSCgWWGG3jeC4v3 bH3fdmzMn0K1vczAmBhdumcjNbujYz6tNkymQu7N2+zWCyZFBNSvj1/YY6Il I2aryhYSrSVY0dSSEhjssBXDfmhH2ce+Aqy1wrUfi08n9EPj6K7YvoiS9Ala Ul2p0Ty3j0qRkfyhBYHUjhaLURFFMiKDSOo8ZDG3Xg3XOS7GOlfrrV/0NmpY ryIciFPM22b7GW0GkHPY0VdDWZso3xelpTSN7OGRsBOEdoib98pW554KUsJ3 +EBHdrWqRUpn3YhrDAx8dtDdQw7vH5M53Qfv1e4/nu3HIDRO8hpG3C/5siDK +KI6gIhROxvOCWKuYtjBZBvpLnroJUCV4dmj97fZ1japbRjRWSSOXWjYg2Hi +jg45h8VxVuWRrQywBhAbQzJMxrd6myuSuipg3obvdXzCbcDv3Q8uoVVUFgj v3w4OJ1R7+q487My3kyYIKPO4F4LV1675Mxs3HiloreinemM5+5B/gPdUy9K kdO3BbWZC+f8gGU93Vv35jmt+9AJ3Z5kGCsMnFUbdTVis9MmKkrGO/Z0XIQ5 hoK6uPX2QSgkC1pTrYWc7lljOzmHa1nbIBrTF/s5xl+UstCs0Gna9lXvHIhP xg5tP9mjYeIjmh3Eq9748Cr3PtkzDDKfcXxvS0BnY/HKNnH8gKOZX9p7BfGU Ur+PgcP7fTqqpC89zPkOvr69MrCL79K/x1wUqPRy3HSDYXqHK1Vr7iXlhK7X o9sDVZZtoWKiL7W6NbFjwoe9pvdmzpfxaTA0M3Q8kti1uJlPRawmovEB35AI 1au7EpuyHjqWBN3s4D+0Sajyd+SNgqJPa4bfZ6N0SpHHoGQPLxGz1SSbCam9 9l4nonU6o9VCFOhGxNVkoCK59yiKWxSxcGM4H08bLU/okFkQ4x7Rpik2Zdh8 BwtAl37vKz4XPnrIv+QNsHj/63fqvtf6/a/b3W73sb7/9btH213Iv/1o6/u7 +1+/xoM3vZoJq29txY+n+69eHx3vHv+jx8yd9LyrIklv0PlEB7FV17+WZGgk AUBsYIg3wWIts2J+da2Se2tOY32O29KzX+7LZIyEpH8rEwcu4nQ0Bx01FcQV FhOvZBWBlWjS4NshRWFjMrRuVdA8zHmQsBjUkbei7YcWQWRee05gJb6ZTJXk 0J4mOiQG99wQ6dpSo4O4X6Eg5SRe3pw1Np/a8udrJty2Lq8i7VLlFPWLwjnJ TnHgDHL5l5+xrqXPh3q5Vr3ch8QCQUtr82qW1Rv2Wy8504g/F5iHLKhkUqDH OeqU7SQdDjUFUbjWHHeQkR+3VfSE8hqk4WBelegpVxZYQFUjiglHKKIDvsS4 GFNIt+QiuwK2T8uqxga5G/BGOYiESt0xihrM1aSYMfVG7ySWtxDPMrphmJUZ urSsXOuf9A8OT/ePX+w+36dzw4aKMWlv//lL/KqMJazTefI2otKhKGRhtuK1 wiappi1QEq6X8IO5bpcrxyt36WZdSrcxI2cZbX4ajCIKYe0wBPQ8pIU7hcAw 6OzYWzHTC7Jq0ufS3hwpjCWouFBz9mAJreZ2GEh7C0o+fI+rbBqUDvpVULE2 jqnwS8Rc3+4km8I1UVR31oP0c9EM1m9rnWXTEYWv5Ff8M6VWWtDMzxUoGjBR dIhNHHK8R0+MOSVgICAc5obSJAYS8St3nmp5UMMBr1jwqzh3wPr1xSidvOUC JkkHJVqz5QkNm+p8/M8/i+PxCvBa589w17+FN2AnjXrNTtmGPedOXeVrtAGt MGN09GW9fjLRvfPaZzVaAmqFbUI3ziM3iAAB2d1SQemMSHOO/nuz4Swn53h/ paYDmMKwuSHPB4m9lVtVGCMAfMhywXA9IdNtg/l3DaI+oajZgPVDNqgoXDnm XG+0RF2u3/6QJytVqM1OxIb6QhQy2Tpd4IJuiAPo2NlQY8JsqYky4UBR/Bhv PnUX873SnBcQMEogAvQoR9bgMTK83y4ZXo1M7do4zPHQTf4uawSBMG04LqiM ShvL1kWkGHWkwzUJaAxXCk+XdKzCajkUI95cte6PM2/jHqMn2TjT+7i6YwSF 2RMajJD56+uJyA3ZjaX7CbSv5ZCPDZZHhBO7J6RZCo5Pr+GaSdSQ6GD8dIWn GQ66kYd82dWY2NiEZlBq7Ki7FYqktTC/dfHRPvga8aKTcvbUuvnReiYNfDqr SgHTdJRXWmqBLpdsOzF1KHcr+cG/4VJhkPlCMk5H7MLXCDi18k0AulKq8EyN i4I0XDUnrifMhRbWzZXXZDG3hXZTwILdPMZ8UhqvQBPh7gxDw6sbZbHOFQTY ojn6ZfJD9gfGz00JVVJcF8Kgvgre5lGQaahGPmSlt0d+6DX/t2QuYg9BVUZF 2L27NJDyaxCdmjkYjYzzhhAbsZo6DStTMzevJwFrgqXuD7lus02vRHsM6u/m Xr50BExjgooxwIkqesUGwob2IIcMeheN3tkN1NDK6is1bXchb3m1JWIPO9GB MdZWNfq81Ryv0GghZwWTqm0Hv1q2jCGEbf366yqD5wyg2tCF9d8SjQBQtt5a qA0EAULU6l1De3uLPZ5A/cJef2LP2L8h1D3nepkYGLILS9tV5yTZZ4tyfkSL hgII+iHtWFva6YgdjwkRZht90bL3edVBppDprTpTzYD2wPG6Ayig5fE6d6At 4r3lTPja3o20V1xawnYueUiyzlUH1/Z0C05L0CCdBeWphqGeHNIpZmLwKJSp 7Sy5A19wRL0OlJsBVHhHJq7bimKqDg/RplhBgdcFStZCwyIZJaLYzlG8nJa0 fIp+SFcVyFsOMfCy2unES7SSqwLZt5HygaOYUiRiU0EvE0zQ5FLf8941nIdr CDI46aLjcbnfYL2R0KtaQaJvBxm2neHNdRobOdN9bl1jHc2wLJeVRisLD26M NaX6sqF3Kfl/vb3px9gCosdjC+zBQh4TQPQIQu38hYFLcoY0cI8XAQBfNARC I1k8WVrLkLUY83I0n6oTuKoBtZIGKugFZk7PBJzm/c/lYkKiIMQaTT31258D J3b11UZcqh0eysj8oLEMzo/PZ6Dk+pY+QSNlsOwKhkout8RYuRYmQU9FcajO P8epaEzZKZlJ4nVw+rI3VJaEYK/pnFLhJh0GRBXKkbQyl0pF1RjCWEPhLaoe mU5YneB+w0wefvc/tGJSp6ZauOWkErWquqR0fjMTtV1FzEb3FPJnRbxo0Z3r vYjW1Nv+eI1QdZE+B9WjT8GVR53DIMMN9DUNX4E4tBcfrkLw6+11HHpgqWY9 +enoX4loJU2in+bgepIPeIMBdQ92W0RaTeiupWoz3URbdmqO1BsHGXQgd6pi TQ4JqMR2OuZOAbxJa5Bfcjt49Opdjgfh2HmGL1oQFTERoqcvXfjU8aZQoE/t Jaq5SyVfYgDWajDdbwjGt+IszCdTwLGEQ0mPlUnAmX/fevPPaYsMCb+prfB0 1mrH0mlYW6DHCXjBcMfWRGL5tmTtGKKA9BKvQqwjZDFK3JuPA1qX0riERqTs 8Pc/D7Wou42Tr0Yyn9RgmG7iinKMbgJ4XMwI5PjUtWK9WgppfTYmpDZGBYye FB7WWE1Q62hLpaONcr2NpNgmFbiNPKJNY99GyMkaKCGZZWQuRFfGWeMs3fz3 7ub/2dr83/3O+YMPPzc//Nz6sPn0w8/3H3xof0D7OJvHNKCOIUD5utkgSLgn TDfzqOuUhcMiy0bA5cHExoP2teUDdeGgzM9xn6JFVFgovxTiP1qmGsqFhSkD Sny0iF4UB619VANGiNBeSM5xB00KHN2ZSqpYBuWVwpuMCtGWcRjMZo6zX/cg 6VLpttiv+w93dLp7go86z/Ql3b+W+H9tbT969IT8v7rbj7pPHpP/Vxc+3fl/ fYUHnWT4MBs7RmflQueX0vghcyHkHo036v7XWYZhplC5TJN1DCK7iRFs+ATL OimfE3EAs4MygWPPyhrpqC3/JN60vksl2G5vT+VTVJ31tbDvs5UH7MRsL7s3 BaBrKr977NcUcKGic6BxqCbuIdU0OTPWDXWGMAYr2WOgrIRX+1WbD5gBPuGf pR1oqEMPlmmbq2qpHbdffIJxUcfUgUi8x3VJVwo+GJoP31s/eWq3UIdj6K/9 zBnR94N+rNazs3vlue2aalJU5/WPj4IuIic+Wrp696jAah3UWVH5Uz9/aydN 62trWE/k5ktyIb/k+1D1Nzo6SGdw8RCePYKr6R7oW1ycaa7YZJsXueW9wyms 8phtf9VGyx7xdavXVWlzAvUP+1YDvVU3fbpFGUTKZoHGjYR/FjnessrXy5Im in5K7cadQrL8sTdcfzkVYIn8f/JEyf/H3e729pPvUf5/1310J/+/xoPy39xt zhHnUORbHcB3ee33L/i4f79vdAGc7q/ykkL4kpdSc5+iwulDYCTlOesbDiyL TUZzqqwGrN5amCPrw1E7vBI3ZznoXfJn9CUWrnv0jXMrh9hvKFhnVnXghUPe dut5++S7YX7bZA0Hemyqn9+oqE15ae+QR+uhOlY3ElcaurHitLHVOhxdZjN1 REq1bWts076auj9oZNaTsn/hSMFur8L3uGouy0aEAqMngxTCv9K7ho7NUJxG PmbDQ7HVMr4qpgAHrK1UTZ1yPs3o/GO/uGyKc3V0YhH324eFiXMKcodiZrOP d6ljZ7N3XI341tyBP9N+bk0JvzEosEdX9rviX8OwaAAi/emajpBNegqisJJq hfA36jNx9lUoRnXmyRnIQTG9bTq2g4mhUzxNyTes+0p2NaQQj0GScLyrVMZP HmqfeUhAVd3iPqJxOnurrt6NLAzUGBmMdmVfXXdHVWwo2Ej0cJnFgnOeDEqb lqpYeGXrgWlV2Td4yt5ww6b5xQrt+jqtv/gasDqNdrBPf1PYTVWkkQQHWofT sQyKPVfQDTPNJ+y3Qif8ST8eoTujDjRcsk9MSu6F+WAO5MtGqOtsNOwkuzDc oN6NZN14fugdhnLjuMR0/Lh4h5MrRxwC/ID1ig8RpBj8FLU+dBbLIIkuQYOu BiWBwq/BS8dNtePNnTGe+Mu4r60wnGsxZ6TGlrJGuy9nC9kZrnglfUUmKfmX ryyHyiNcW6uxOtvZSLa1RY2sxIZsE34mzZD0AAkEKpakagctrM/YEJUjH7kq qipFRmLjbCIf0fZf1ivUfDpWwfeD52x8wwGfWrEeiligUrFxIqYDLiI40XRe Xmt0luEGOjDhQB5iwJuJ9VWaug2h5T/eir5KoMbriFGVhjPZwpp03CC/mvF3 DMeXERA43m+NFdvy9johcYBWaI0OaEsvatDBd+sVuL7c0sa9aE7X0VfXflaZ LoF66lL891buV3jwmHxnmn/RNpas/7rdLbX+g9Tvt+F3t/td9/Hd+u9rPBsi eEPKsZIoTADyDHW4154ZoSOV6PZKcebVmd9JYs+Q07rRnM15URTMYkEoqtNB aS/hLTb58YI+4l612UuTyYMeTkdSS+XnIX9+34Oyflom0zafBqu9FHls2y3Z gWfpzO/AlSjVTm57upSb6zpYN+b3GnOL5S7Yi5Dyz17S9PK2opnfuvU2FwIx 0pltEZF582kAsb83Ed89n/wYufsFdwAX8/9Hj7vb32n735PH3z9C+9/j77t3 /P9rPBsqFoI52YZO/LQPQhuBv/WmEHtlCDb008H+8e7x85/+waEg0HKzZ5pV 8QZMOFERf+B0D7M6H/6WV9dmreyFKlDxo2pfX1HAidpnHZrJqf9ExVyu5baB DCNJdDGSU5e+Mumquq6V0WnBCp27lpwqbUA/tyV9g5XGpT0EYT6FsHa694Y9 SfxEqGFyS4P1uejA3U0ObjSXt7hW+8f+CR4mhz+Nb8gOh4u0bHSb3EK5wyNM OzzykibFGkWGw0T6gekc9/gCr7qamW1dEwXusKj2sIJsGDRFCwu3JVW9cKRl rFofvDNbjBjdlW5lurnO6FwNakuUrALBmbvuzAkgMfs667U1ZT4ZjObDrGx6 ayNu0qxXHIuq3l12IdI2aX/Gw6dxMcuSq2yCccj5RoF6qGUNGxnjAHEHuBWP e/nZ0FheaZU+scdnBQizjO4YVzYlcyygYIspe73V2iw7JjQ15ULoh97iERMc 42vMuBzI75uixXLbv9BKvTlHi0wndd/1ire2Cl8/zSjeiKYJcfiL73NAnZrd idFRU9OnSw+HR2aRq8MH1hs6prylvRXMiZsdqTkwIQzxm8vO3GnA9sdTj5Zw 5aDW6mpHOGy149Bzjt8DhYo2d80vo2l9ituNZ+eFoxYRqE0bS4cKDUp4RnCY 3rYTFTbScEYgfzWC6PYxwCA2hFNaPGlPwOx9Sm4qO0lf/SRzStPAYIyAOqd3 LIrCNg4L5FwUxBxP2HHONp6mg2GZj/EQI+ShK1VUNhMw3SVgIBvVnO6zqSxk jPnH/snHEt1v7jAzlRAVCkgixiG3M9ZXNO5lQSTVSB5YoujoaHyW6pGym5ae 2H2kxjudIwBI/FiMJgW5RA3QjdjMRPa2i00IKUNMu53ArCk7uhFhluWrZne4 FtsLR1Na3h11dkFNYN4R0AKI/YFUVdFeqIuJRBjAJT3qW/pQMSqJxkQFaGU4 Ux1HXeiUWnD9TWwU5opcuG1xc5gwGABTaEQCmU7IJGEBWcY6tDuvNrb2MVsZ mV/j9PYCR2zLA92DQJdeEMFXxvCF2mvWXN2izOfE0LXAdHU/6IMHOZX5WN6w tGMh+3M9Qi4nK/tzgFF4JujFTGU5i7hX/tDpdJ6irxKPJPEMGTKE4jtyb9Fm ptwQ0GFuUu00GpYiOPCmy+/oG9+RV1bDYs5+5/i1czMDbbYJAAAIP0/IEYwr bfuQqL2k5QhWwSg5jidXRrGcEnW4wW336b1yabuGv9Baq+nxGeq7zY/6OOVH n0+eErij0uhzfPc+Om31+xdpmZX6hZlx4zzCZALcxQEgzDUJ1lOfPS2nYOmk b2D2pgbbwsTCrF5Qd9DZAFcRwMd4eiwT6jnuZtfOyDuLzKa5BtxtRqEuCJ+r yLooC/XfjjOvnlcYaI7rGBvpzzugDNTXHNFPwp2J/uyK3wU+TatpAbreU1+A Chmna2zruSqkXMBzaWUJ52AuUb4Q6qvyNPER6umVDgieUPtaUzICqQNaHOZ6 DPclQkWPCouWryZITLPM1rnmAB18nJTQtbqVGqLXJrTVdE5SkZGrPaS7HfSd ZyssIdtJNqLAJavMGdFpVQpwp379lkWn7iyvOmX9vjFCf1+FXEABuI8O00/F ESzDa9vktC2bkmJZXLgioFsyAJidEhnhLl+nE8ORcfBHQDcYZltaZbflfLhZ vq0OOOWPQk509XlAF8uPOuzC0BsmejL28ArLmHo2+WJXQ/FtjlJrTUEZ29gS vOoodopEArR0vcXzoDYRaHFBeDcVrTIjlG+fNyNcl0PaQTTXbFCYetkwHU7g 64Y6XifMXLBrJAqdWq9D55cXOujEs/w8Ng3pdqja6mNLTsPugpnKaBun0yad PnBB8ucy8MzoRDYHGeiMLp1iAPSjhnig7EDo1fRW64tsFAKO/B4jrjEF4QTF C/DQClqMs4tieEvmIaolTfQ5ipa5MbZBCucMT5Bm77LZLftNFZeXqHdiIV4T ogVMk7a91ZjCXyWDa4xhq223E473xlNyj6C45kO4/hRRjEnMmMUGCX+6PEQk c48/jVXZWSJgWJFdlYE5rznWb+wRTcBP5WEf0ac6H5OdEvdFrSy8sYzmwEEg +YKptrnRajUVF6v1DF1cEUaAoh+eBYyT9M/fItMtFryrDT1IaqyFP8cKadBq xQxiVtAMfuGj2b9afqLR67QhxlRsWqxixgfgVMBmEcPxoXEHXqyW3Yci7q0z KpAbXSanxQZpzvCRrgQ058cMZVLooGAW/4gZZMDjkZRNfwo0S0kbyeUI1Ha1 v8U3x1+KLq7ZCukmTyzNogeK9TlBXvnGdhf4LuRPjZIs7oWogTKd7H3FQQo7 ph0/qotu1w+hSOPM8OgsqxA7MZ8RhQ/T9iIXeE6MbqEJqejIy2XLxllWzkfm tAGPVcCQGQbPvwyRQXRKowrBbVhIN4S/OkVDTEcjvU8i7ueicI8T3PChTJQh rzqimjQZ5++TcZZOSnOt7KRI8CqMqtBbyx0fQzVjKcO3fG0buBgyOmKO7VTE bVI4cleon2RENYc3rBl7kVlVop12a71TBHZMGuLUNLsDCFw5DEvqoisqYWY+ mYO1tN1TU7RINW3ykdFWQ8h1HTVvqQDkPRItztGN0lzcio7FaYW0EeOUi2Py OSyUeSHnCzBDmZWD9ChDQUIb/IYjqQaclZZjO+IIMuHClObJxrVVxLhG56m3 9jCRDMXVcDx+Juye+vXNBqt7CZEywsVmEr2ucnbCtBKAR/G+QU1W7oOrBRVF iExZ4/pRaBc3Irzkpy+AxHVqzuLHrE24wzUlgD9HFkxm9O16SX9avFyyN004 9dhTd3xShxlgrgLKkFKPCp3NVFRZzx4a4uzkzYKb0nQkCGkfZjleBYe6gwqk xPcHmXr4gYS37PaAwRxMojkWBqwVT4bhsgIpEbetkBFzvHAEQJ8O024fqluw otN35Q69bxKrYixEpm59peikxtaKJvR/Ks70GNapSJLkTCUyBnTAfkjYchsc +2vothGEgoi3DgXTtoSCvtQV2JsVoHDakHrASgtndaugFnJB9GEeEzjgvuLf TZtPnwEJ9tot7Jbm/rXsdSz6zIaVsDzmwcpEVCQ/OKLqlRQzfAsjLegdeYR4 QdtesvlU18ZZBXdwthDMFn5Ae69LJZqMUihxRZDC+zMrbOSjUT1wjBr/fLOh zklr7rsKowyf09J7XEu9yhbtSBj4wudtP92vxYZuW3Ss9aOcWKpVdp/N6PwC +Pn1a+4U6JtJ6lsFwYkSOufqVuldDePtLMsZpGndd0ZtSh8NovcXAIoOfqbc ldv65lvtp4XnusqqZHOq1TQxgu58Glu7GqNEdOPxl1+X+/o5C4CPGewff/zx 9xhsajeyjbM7uf0YlmOCQLHm30gntw08UowLDovzFcwx3aVYthZKGnl1Nb26 6BkWZzMM8/NjdEDik/7jK1bht5eMM8b3W+PIPUFXXsIrqtFD1Esjdjb24tSO ZOoGVewG17HGphW7PpCad8+D2fiL+4W0e5lTQHnERdoxvhUWhwpxt3gnGd5M lFdifsqa1YRnt32/YrvH/+k1K9d/v2q5f/7plZsDBGsCVdpltlKhmmgJbAac l7TYpDBU4SdQE/Sd3KT0ml0jfZuDCjI8JC6hjWionS4xotWyuEY0tf0z1AeK VQwyFWKUOH/2Xtsg8LuxyOj7zNnpW+USQZpQ3OGxQZD3eGXVZJjOhj5xl2t4 S8SO4jatNd64pw8GjwQ0J/Ai8oBum/NzwFdOptBfoSrwO2fhMJyBPPhZrVTp fAEgoGjeJ+Wul5DbDalt8KO3JqYR1WQMB5DQRv3WW2tTcY4bS9X/QtXQNRm/ YmV2ZU3VCVM8VUg+P1SMcaFL+r4OfmF9BFGXV1ITF1nI1m54R0htNZlQ2FWK URkE9fuimZqpyevWmjPvKJPyH2r6mdstk5vmP2Vmp7JFefWko+zGv0ZW1DIX k4/TqbmcHPp7meGCVfsha0QUl0mW4sUtt3TUiH3AlUuvsFOxk/ePa8bNGeNJ rCVqUj2bX15mM5pWwOLxdxMEdVukE5k5PBlvqLM5LBtl+wZo2godNCp4k2lZ 2dzFMNOtSYHSodtLBpAqai4Q3vecfeufNkFvsfQwMIb9vD8a5VNQoThJv9n0 F7niygpQsrNc5u8J5zbXqEhV7VudLVEab2t1+hks7SDK7aOty2HjSOGMNUMi AcQpNgFA2WoMR6RZaz8XzHEg70v79RWsXoqhaVEbo0UGK7b4CwOlZkQAJM10 FELMd8OwehxGsNkVBGUZYi9pNOz30xlMl4t08HYpio2oAM4mKjYagPduc/z9 GI1KnOc9G5i2qIJf7w4E/4953qWzvJiXXzQExNL4f9v2/vetLt//3r2L//tV HhP/gdaL5gYLcfwB5Z+ikprpqHQDPpS344tixJ6tNqAAmmtHeH3odF6Fwj9g y/HUDJYbC8qCfAD9LJRk7gMvQ6nqCgTcUQjWO43UejmNFSmrcfg7BSzuR5PH 6WgUTc3eT2fRRAraHk0dZvFqcW8vmng5Km6iiXRVSTRV3+AazcBrgngy2s+i qXw8ON5heyVBKFmF24tjOhtEE3mRtqDbANl8MoxDfhlNosuk4mNRxIe/mt0u 6A3aW9D9DvhWkOzmeRVMQAU3iALoXixtUlTRNMQNcI8SL8wNp/aLaYz0Q9/f A0piaQhkLK28zi+reMlZXl1HU/EKoODwpAM8Dh6aYcVNFm6oKoJVjdLxRYTn 0F2u4drK+UU5mOXTKsqVdIZgIqqTcezH6sShjqWhOXicvg1DSybFSCeBG8fq 1HcWcZqQN3QliS9u9g/3Xu0e/3n/OFQVXl4S/P7m1bNwiZPT44PDPwXL7P/t 5cFhsLqDw739w9NQyt5+LOXl690gAMeR7y9P/vIsmD/y/fnRy6PDcMKrV7vB ru+/Ogh9f/3yzUno+6uDw3DCyWm4Cycvd09+CiX89Vm4wO6r1+FRerl/Emz5 T8f7u6fhIvt/ebP7MjhER8HxgZafR4bu2e7zP//lzdFpkBZePjvGy+xDIxVN 2f9LFLpDaCaWhliIJipURNNPD17uBYF5fnD8/M2rFy/3/x5u88XpyU8HL4KI OT7400/x1L2jN88A5AhxHL0OYgZvqDk9+vN+kJQP+5QUJIXD0/7Rixcn+6c1 NsLhy2pqK0tIDHcVDpSVltU2OY9wJrLOHeDWye+t0989d8/dc/fcPXfP3XP3 3D13z91z99w9d8/dc/fcPXfP3XP33D13z/8HNpXCOgAYAQA= --1658348780-1058415798-946906158=:412-- From paul@prescod.net Mon Jan 3 23:26:09 2000 From: paul@prescod.net (Paul Prescod) Date: Mon, 03 Jan 2000 18:26:09 -0500 Subject: [Types-sig] Re: syntax and compile/run -time References: Message-ID: <38713011.C011506F@prescod.net> Greg Stein wrote: > > Dividing the language into pieces is simply dividing it. I see no rational > reason for doing so, but many reasons to keep everything clean and > integrated. Dividing the language is inevitable. The question is whether to do it "up front" or subtly through cryptic error messages. > > This, for instance, is not legal: > > > > a = doSomething() > > b = typedef a > > It certainly can be legal. But the type-checker would say "I can't help > you here." I don't want the type-checker flagging valid code with warnings (unless you turn -Wall) and I don't want it silently ignoring code like the above because it doesn't understand it. You are setting up three levels of conformance: 1. Checked at compile time. 2. Seems like it *might* be checked at compile time but you get a warning telling you that it doesn't. 3. Checked at runtime. I vote to remove the middle option by clearly flagging which things fall into the former and latter categories. Also, what do you do about forward declarations? They are not a problem with my syntax. > I believe this distinction is unnecessary and is on the wrong track. I do > not believe there is a requirement to create a partitioning of the > language and its semantics. It is much better to leverage the existing > semantics of Python than to create a whole new set. The partitioning is not avoidable. Some objects are evaluated at 12:35 when the static type checker runs. Other objects are evaluated at 12:27 when the code starts to interpret. You can hide that and then issue warning messages but all you have done is *hide it*. I think "be explicit" is more Pythonic. > Specifically, in your example: if "a" or "b" was used in a later typedecl: > > def foo(x: a)->None: > ... > > We Could issue a warning that might read, "The type information > represented by is not available at compile time; compile-time checking > is not enabled for parameter in the function , nor for calls to > ." > [ well, some suitable word-smithing is needed there :-) ... but you get > the idea. ] I don't think word-smithing will help. The problem is that you've taken an unavoidable, temporal distinction, papered it over, and then re-introduced it through warning messages. That is unavoidably confusing. Paul Prescod From gstein@lyra.org Mon Jan 3 23:58:04 2000 From: gstein@lyra.org (Greg Stein) Date: Mon, 3 Jan 2000 15:58:04 -0800 (PST) Subject: [Types-sig] Re: syntax and compile/run -time In-Reply-To: <38713011.C011506F@prescod.net> Message-ID: On Mon, 3 Jan 2000, Paul Prescod wrote: > Greg Stein wrote: > > Dividing the language into pieces is simply dividing it. I see no rational > > reason for doing so, but many reasons to keep everything clean and > > integrated. > > Dividing the language is inevitable. The question is whether to do it > "up front" or subtly through cryptic error messages. You are presuming that the alternative to "up front" is "cryptic error messages". Again: this is FUD. I do not believe that smooth integration necessarily leads to cryptic error messages. Not at all. I am certainly biased, but I believe that my syntax suggestions and run-time and compile-time models are quite clean and natural-feelings extensions for Python. And I see them as *extensions* rather than a distinct wart glued into the existing Python. > > > This, for instance, is not legal: > > > > > > a = doSomething() > > > b = typedef a > > > > It certainly can be legal. But the type-checker would say "I can't help > > you here." > > I don't want the type-checker flagging valid code with warnings (unless > you turn -Wall) and I don't want it silently ignoring code like the > above because it doesn't understand it. I agree. I've been presuming that a number of warnings will only be flagged with -Wall. Whether my explanation on what happens with your example truly issues a warning or not... fine-tuning. And I never said that it would ignore the above code. I think the type-checker will be well-aware of what is happening and can provide several options for us to choose from in its response. Your syntax with a "typedef" statement attempts to avoid the problem by restricting the syntax to an arbitrary subset of Python. My proposal allows people to use similar, limited syntax *or* to use complex stuff for complex, runtime operations. > You are setting up three levels of conformance: > > 1. Checked at compile time. > 2. Seems like it *might* be checked at compile time but you get a > warning telling you that it doesn't. > 3. Checked at runtime. > > I vote to remove the middle option by clearly flagging which things fall > into the former and latter categories. Fine. We can declare your example "illegal". I was simply trying to state that it *can* be legal. Look at my words, quoted above. I was attempting to move it to #3 -- runtime only. Even though we move it there, the type-checker can tell that the user may have thought it fell into #1, so it can issue a warning. But hey: this is what Guido calls, to paraphase, "appropriate error messages." Some fine-tuning is all. It does not invalidate my original point: there is no need to separate compile-time and run-time syntax/semantics. > Also, what do you do about forward declarations? They are not a problem > with my syntax. I have said this repeatedly. I will repeat one more time: ==> "incomplete" classes and interfaces may be declared Effectively, this enters the name into the namespace for usage, but the actual interface is not yet known. As an example, we do the following in C code: struct foo_object { struct foo_private * priv; struct foo_object * next; }; struct foo_private { struct foo_object * owner; int data; }; These are mutually recursive structures. To model the same thing in Python, we do the following: decl incomplete class foo_private decl incomplete class foo_object class foo_object: decl member priv: foo_private decl member next: foo_object class foo_private: decl member owner: foo_object decl member data: Int Technically, the "decl incomplete class foo_object" could possibly be omitted because the "class foo_object" will automatically create that, then fill in the final interface at the end of the classdef. In the above scenario, all names are available and their semantics are well-defined at compile-time and run-time. The human-level semantics are well-understood, as we have analogies in other languages. In this example, there is no separate, distinct namespace. We continue to use the standard Python triplet of namespaces (local, global, builtin). Everything looks and operates quite normally. The PyClassObject would modified to allow for an update to its bases and methods (at the end of the classdef); an incomplete class would raise errors if somebody attempts to instantiate one. > > I believe this distinction is unnecessary and is on the wrong track. I do > > not believe there is a requirement to create a partitioning of the > > language and its semantics. It is much better to leverage the existing > > semantics of Python than to create a whole new set. > > The partitioning is not avoidable. Some objects are evaluated at 12:35 > when the static type checker runs. Other objects are evaluated at 12:27 > when the code starts to interpret. You can hide that and then issue > warning messages but all you have done is *hide it*. I think "be > explicit" is more Pythonic. I don't believe anything has been hidden. Instead, I am trying to use standard Python language, conventions, syntax, and idiom as a leverage point for a new type declaration/checking system. Introducing new sub-languages through a single "decl" statement, or adding a new namespace, are both items that I believe are too divergent. I also believe they are wholly unnecessary and have provided explicit solutions to avoid them. Yes, "decl" is needed, but it shouldn't be a trap door, catch-all solution that some people want to view it as. > > Specifically, in your example: if "a" or "b" was used in a later typedecl: > > > > def foo(x: a)->None: > > ... > > > > We Could issue a warning that might read, "The type information > > represented by is not available at compile time; compile-time checking > > is not enabled for parameter in the function , nor for calls to > > ." > > [ well, some suitable word-smithing is needed there :-) ... but you get > > the idea. ] > > I don't think word-smithing will help. The problem is that you've taken > an unavoidable, temporal distinction, papered it over, and then > re-introduced it through warning messages. That is unavoidably > confusing. Oh, bunk. I was attempting to make it actually *work* for the user. In your model, it would just punt. You are creating artificial limitations, and you're doing it through a partitioned syntax. Neither is desirable. If somebody says: MyComplicatedType = doSomething() def foo(x: MyComplicatedType): ... Why should we prevent them? I say, let them do it, but tell them they aren't going to get compile-time assistance with the construct. Keep the syntax clean. Provide flexibility. Let the developer know if they have done something suspicious. Cheers, -g -- Greg Stein, http://www.lyra.org/ From gstein@lyra.org Tue Jan 4 01:28:14 2000 From: gstein@lyra.org (Greg Stein) Date: Mon, 3 Jan 2000 17:28:14 -0800 (PST) Subject: [Types-sig] updated web pages Message-ID: I've set up a new group of pages for my type system proposal stuff. There are links to my proposal document plus all of the code... http://www.lyra.org/greg/python/typesys/ Cheers, -g -- Greg Stein, http://www.lyra.org/ From gstein@lyra.org Tue Jan 4 02:41:07 2000 From: gstein@lyra.org (Greg Stein) Date: Mon, 3 Jan 2000 18:41:07 -0800 (PST) Subject: [Types-sig] updated proposal Message-ID: I've updated my proposal document at: http://www.lyra.org/greg/python/typesys/type-proposal.html There have been some minor refinements to text, examples, and a small syntax change to type declarator (had to add '(' typedecl ')' to deal with some binding precedence issues). I've also added a big section on what I see as open issues. I've listed by biased :-) thoughts along with my understanding of the counter-proposal. Comments are most welcome! Cheers, -g -- Greg Stein, http://www.lyra.org/ From gstein@lyra.org Tue Jan 4 03:25:15 2000 From: gstein@lyra.org (Greg Stein) Date: Mon, 3 Jan 2000 19:25:15 -0800 (PST) Subject: [Types-sig] parameterization and syntax Message-ID: In going over some of this stuff, I've been thinking more on parameterization so that I can nail down a concrete syntax proposal. Here are the items that I believe we'd like to parameterize: * classes * interfaces * function signatures * builtin types (such as List) Classes may be parameterizable by virtue of parameterized interfaces. Builtin types (such as List) could also be parameterized via interfaces. To constrast: non-function variables/attributes are not parameterizable in isolation, but may be parameterized via a parameterized interface. SIMPLIFY -------- To reduce the problem set, let's start with a few simplifications. First: a class definition and an interface declaration are nearly identical in syntax, so the two should also share parameterization syntax. For example: class Foo(Base1, Base2): decl member a: Int def b(x: Int): pass interface Foo(Base1, Base2): decl member a: Int def b(x: Int): "doc string only" Second simplifying assumption: an interface can be associated with builtin types. Interfaces will be expressive enough to provide for the parameterization of things like List, Dict, and Tuple. RESULT ------ We are now concerned with parameterizing interfaces and functions. INTERFACES ---------- The syntax that has been introduced seems to work well. Notably: interface (_X, _Y) Foo(Bases): decl member a: _X decl member b: _Y My only comment here is that usage places the _X and _Y *after* the name Foo. For example: def bar(x: Foo(Int, String)): ... I think that I would like to suggest a little borrowing from the C++ template syntax: interface Foo<_X, _Y>(Bases): ... def bar(x: Foo): ... This makes it very clear that we are not performing a function invocation when the concrete interface is constructed (in the bar() definition). This also allows us to place the _X and _Y parameters after the name "Foo" and to even use the same syntax between declaration and usage. [ note that we shouldn't say: interface Foo(_X,_Y)(Bases) due to grammar problems. it may be possible, but rather ugly grammar-wise. ] There is conceivably a parseing issue with things like: x = y ! SomeType When the '<' is seen, it is unknown whether it is part of the typedecl, or part of the outer expression. It would be quite easy for the parser to make a call, but it is also arguable on whether that ambiguity should even be left in (note: the Python grammar *does* have some ambiguities that Guido resolves at compile-time). An alternative to the < > markers are braces: interface Foo{_X,_Y}(Bases): ... def bar(x: Foo{Int, String}): ... There are no ambiguities here because braces are not operators like parentheses or angle-brackets. [ I just realized that: x = y ! SomeType(Int) has similar binding precedence issues. is (y!SomeType)(Int) or y!(SomeType(Int)) ] Also, braces are never allowed to occur just after a dotted_name, meaning that "dotted_name { typedecl-list }" can always be correctly parsed. OPEN ISSUE: parenthese/brackets/braces. OPEN ISSUE: placement of parameters in the "interface" line Operation of type-parameters: in the above examples, we have declared _X and _Y to be type-parameters that will be bound to a particular type. The operational model is that the names are in scope for the duration of the "interface" suite (this applies to "class", too, remember). Any type declarator within the scope may use these names. A "parameter-reference type declarator" will be constructed (at compile/run-time) which holds the type-parameter name and type-parameter index. When the interface is made concrete, the reference evaluates to the concrete type declarator passed. FUNCTIONS ========= Functions have a number of type declarators associated with them: argument types and a return type. Being type declarators, they can use any type-parameter name that is placed into scope through a parameterized interface. However, a function may need to be parameterized *outside* of an interface, or to use additional type-parameters within a parameterized interface. Specifically, consider the following function: def Add(x: _X, y: _X) -> _X: return x + y The question here is, how do we introduce the parameter name _X? ==> we don't want to use a notation similar to interfaces because we won't be creating a concrete form of the function -- we're just saying (in this example) that its argument types must match and the return type will match that. ==> we have to declare _X somehow so that we can distinguish between a NameError and an intended parameterization. Note: a type-parameter name must appear at least twice, to establish a relationship among the function's type declarators. Something like: def whatever(x: _X)->None: ... does not make any "rational" sense. ==> we could introduce a token, such as "param": def Add(x: param _X, y: param _X) -> param _X: This is a bit wordy, but works. If we want to cut down on the words, then where does the token go? The first occurrence? That creates a weird "the first one is special" rule that doesn't normally occur in Python. ==> we could go ahead and use braces: def Add{_X}(x: _X, y: _X) -> _X: and just live with "but you aren't supposed to call the function by saying Add{Int}(x,y)". ==> other ideas? I'm out Note that it will also be important to mix parameterized interfaces and functions: interface MySequence{_X}: def getitem(x: _X{_Y}) -> _Y: "doc" f = MyClass() ! MySequence{List} l = something() ! List{Int} z = f.getitem(l) ! Int OPEN ISSUE: how do we declare that a name is a type-parameter in a function definition/declaration? Cheers, -g -- Greg Stein, http://www.lyra.org/ From Tony Lownds Tue Jan 4 03:59:13 2000 From: Tony Lownds (Tony Lownds) Date: Mon, 3 Jan 2000 19:59:13 -0800 (PST) Subject: [Types-sig] updated proposal In-Reply-To: Message-ID: I just tried to compile Python 1.5.2 with your syntax changes. I got pretty far but your syntax needed a bit of tweaking. Here is a tweaked version of the "type declarators" code snippet that worked[1]: # first NAME should be: 'member', 'var', 'class', etc decl_stmt: 'decl' NAME NAME ':' typedecl typedecl: item_tdecl ('or' item_tdecl)* item_tdecl: param_tdecl | list_tdecl | dict_tdecl | func_tdecl | tuple_tdecl param_tdecl: dotted_name ['(' arglist_tdecl ')'] tuple_tdecl: '(' typedecl (',' typedecl)* [','] ')' list_tdecl: '[' typedecl ']' dict_tdecl: '{' typedecl ':' typedecl '}' func_tdecl: 'def' '(' [varargs_tdecl] ')' '->' typedecl varargs_tdecl: (arg_tdecl ',')* ('*' ':' typedecl [',' '**' ':' typedecl] | '**' ':' typedecl) | arg_tdecl (',' arg_tdecl)* [','] arg_tdecl: typedecl The changes are: - the grouping alternative in item_tdecl conflicted with tuple_tdecl, so tuple_tdecl handles grouping and tuples - arg_tdecl cannot have a name, just a type. I don't see a way to have: arg_tdecl: [NAME ':'] typedecl in python's parser, because typedecl can start with a NAME via dotted_name) so the rule makes an ambiguous DFA. You can have: arg_tdecl: typedecl [':' typedecl] and constrain the first typedecl in the AST-consuming code. I'm going to continue to add the syntax you've laid out. I've independently been writing grammar for in-line function definitions stuff so that'll be the next piece. However, I'd like to hear from people who have an opinion if using a modified build is even adviseable. My recent experience suggests it is not because when you change the grammer you often trip up the compiler and end up with a broken python. If it *is* adviseable, then the thing to do is to completely specify the syntax and then make a Grammar and compile.c that works with current syntax (ignoring any type declarations). If it is *not* adviseable, then the thing to do is to make check.py go through an interface instead of accessing the parser, token and symbol modules directly, and look for another way to turn Python programs into parse trees. Either way I'm pretty interested in getting check.py to work on a modified Python syntax, if only to test ideas. -Tony [1] test run - the python binary was built with a modified Grammar/Grammar habib:~/src/Pyhack/Python-1.5.2> ./python syntest.pyc Module readline not available. >> decl mamber sprintf as def(String, *:[Any]) -> String (file_input, (stmt, (simple_stmt, (small_stmt, (decl_stmt, (NAME, 'decl'), (NAME, 'mamber'), (NAME, 'sprintf'), (NAME, 'as'), (typedecl, (item_tdecl, (func_tdecl, (NAME, 'def'), (LPAR, '('), (varargs_tdecl, (arg_tdecl, (typedecl, (item_tdecl, (param_tdecl, (dotted_name, (NAME, 'String')))))), (COMMA, ','), (STAR, '*'), (COLON, ':'), (typedecl, (item_tdecl, (list_tdecl, (LSQB, '['), (typedecl, (item_tdecl, (param_tdecl, (dotted_name, (NAME, 'Any'))))), (RSQB, ']'))))), (RPAR, ')'), (RETURNS, '->'), (typedecl, (item_tdecl, (param_tdecl, (dotted_name, (NAME, 'String')))))))))), (NEWLINE, ''))), (ENDMARKER, '')) >> On Mon, 3 Jan 2000, Greg Stein wrote: > I've updated my proposal document at: > > http://www.lyra.org/greg/python/typesys/type-proposal.html > > > There have been some minor refinements to text, examples, and a small > syntax change to type declarator (had to add '(' typedecl ')' to deal with> some binding precedence issues). > > I've also added a big section on what I see as open issues. I've listed by > biased :-) thoughts along with my understanding of the counter-proposal. > > Comments are most welcome! > > Cheers, > -g > > -- > Greg Stein, http://www.lyra.org/ > > > _______________________________________________ > Types-SIG mailing list > Types-SIG@python.org > http://www.python.org/mailman/listinfo/types-sig > From Tony Lownds Tue Jan 4 04:51:40 2000 From: Tony Lownds (Tony Lownds) Date: Mon, 3 Jan 2000 20:51:40 -0800 (PST) Subject: [Types-sig] parameterization and syntax In-Reply-To: Message-ID: On Mon, 3 Jan 2000, Greg Stein wrote: > > interface Foo{_X,_Y}(Bases): > ... > def bar(x: Foo{Int, String}): > ... > Hmm, how about an "of" keyword: interface (_X,_Y) Foo(Bases): ... def bar(x: Foo of (Int, String)): ... That seems much more readable to me. A caveat is that handling a single, tuple argument is tricky: interface (_X) A: ... interface (_X,_Y) B: ... def f(a: A of (Int, Int)): ... def g(b: B of (Int, Int)): ... It'd be nice if the "def g" above could work but that means you have to look at the interface to see whether (Int, Int) will be assigned to one type name or split into two. Anyway, there are the pros and cons. IMO the extra readability "of" affords is worth it! > ==> we could introduce a token, such as "param": > > def Add(x: param _X, y: param _X) -> param _X: > > This is a bit wordy, but works. If we want to cut down on the words, > then where does the token go? The first occurrence? That creates a > weird "the first one is special" rule that doesn't normally occur in > Python. > Using $ instead of "param" as a marker would make it less wordy and easier to put in the grammar but also more perl-like. IMO the param marker should go everywhere that its used, just like you show in the example. > ==> other ideas? I'm out how about simply not using parameters: def Add(x: Any, y: type of x) -> type of x: "add two numbers" Ignoring the "of" business, you don't introduce a name for a type parameter - you just refer to the type of another formal argument. > Note that it will also be important to mix parameterized interfaces and > functions: > > interface MySequence{_X}: > def getitem(x: _X{_Y}) -> _Y: > "doc" > > f = MyClass() ! MySequence{List} > l = something() ! List{Int} > z = f.getitem(l) ! Int > This can be worked into the scheme above if one can reference the types a parameterized type is bound to via special member variables: interface (_X) MySequence: def getitem(x: _X) -> _X.__type_param__[0]: "doc" -Tony From Tony Lownds Tue Jan 4 05:19:05 2000 From: Tony Lownds (Tony Lownds) Date: Mon, 3 Jan 2000 21:19:05 -0800 (PST) Subject: [Types-sig] updated proposal In-Reply-To: Message-ID: On http://www.lyra.org/greg/python/typesys/type-proposal.html: > Typedefs > [...] > > Question: > typedecl allows for an 'or' between items in the declarator. Does > this create parsing problems with the 'or' in a test rule? For example, > will the following be unambiguous to the > parser: > > a = typedef Int or String > > How will the parser bind it? > (typedef Int) or String > (typedef Int or String) > I tried the typedef syntax and tested it and the answer is (typedef Int or String). From Greg's email titled "[Types-sig] parameterization and syntax": > [ I just realized that: x = y ! SomeType(Int) has similar binding > precedence issues. is (y!SomeType)(Int) or y!(SomeType(Int)) ] I tried this too; it binds as y!(SomeType(Int)) -Tony From paul@prescod.net Tue Jan 4 04:39:45 2000 From: paul@prescod.net (Paul Prescod) Date: Mon, 03 Jan 2000 23:39:45 -0500 Subject: [Types-sig] Re: feedback: PyDL RFC 0.4 References: Message-ID: <38717991.6B2E0B58@prescod.net> Thanks for your feedback. I will need a lot more before we are done this thing! Greg Stein wrote: > > Wouldn't these be called "abstract interfaces" or "parameterized > intefaces"? That seems to be a more standard terminology. I don't like "parameterized interface" because Sequence(Int) *is* parameterized. I need to distinguish Sequence(_X) from Sequence(Int). Abstract is a little better, but we aren't dealing with abstract classes as they are known in C++ or Java. > > Typedefs allow us to give names to complete or incomplete interfaces > > described by interface expressions. Typedefs are an interface > > expression re-use mechanism. > > typedefs are also used to assign names to things like "Int or String". > > I don't see "Int" as an interface (even though it probably is in *theory*, > it doesn't seem that way in layman usage). It makes the spec much easier to read and write if we think of them uniformly as interfaces. Else we must constantly refer to "interfaces and thingees like Int and String." > And I still don't understand the need to specify that *two* files exist. > Why are we going to look for two files? Isn't one sufficient? One is where you put your hand-written declarations. The other is where the interpreter dumps the declarations that it extracts from the Python file. That way you can use all inline declarations, a separate file or *both* with no danger of having your hard work overwritten. > In the above example, we have three interface objects. One is available > via the name "foo1" in the module-level namespace. One is available as > "Bar.foo2" (via the class' namespace, and the class is in the module > namespace). The third, foo2, is only available within the function Baz(). We're making a static type checking system. I don't see what runtime definition of interfaces in a function scope buys other than confusion. If we need to have interface decarlations in random contexts then we should differentiate compile-time available ones with a "decl" keyword prefix. > I do not believe there is a need to place the interfaces into a distinct > namespace. I'd be happy to hear one (besides forward-refs, which can be > handled by an incomplete interface definition). A static type checking system exists to precede and constrain dynamism, not to expand it. > What does "builtin" mean? That these interfaces are magically predefined > somewhere and available anywhere? Yes. > Note: you probably want to remove the plural from "Modules", "Methods", > and *Methods. Fixed. > What is the "Null" interface? Is that supposed to be None's interface? I > don't believe that we need a name for None's interface, do we? And why > introduce a name like "Null"? That doesn't seem very descriptive of the > interface; something like NoneInterface might be better. Okay, I'll just use None. > > Certain interfaces may have only one implementation. These "primitive" > > types > > are Int, Long, Float, String, UnboundMethods, BoundMethods, Module, > > Function > > and Null. Over time this list may get shorter as the Python > > implementation is generalized to work mostly by interfaces. > > I don't understand what you're saying here. This paragraph doesn't seem to > be relevant. It is crucial to the distinction between implementations and interfaces. Certain types do not have such a distinction so you cannot just implement the right attributes and expect it to "work". You cannot make a new class that Python treats as an Integer. You cannot make a new class that MFC treats as a window handle. Here's what I say in my current working version: > Sometimes there exists code that is only compatible with a single > implementation of an interface. This is the case when the object's > actual bit-pattern is more important than its interface. Examples > include integers, window handles, C pointers and so forth. For this > reason, every class is considered also an interface. Only instances of > the class and its subclasses (if any) conform to the interface. These > are called "implementation specific interfaces." > > Note: The Python interface graph may not always be a tree. For > > instance there might someday be a type that is both a mapping and a > > sequence. > > In the above statement, you're mixing up implementations (which can use > disjoint interfaces) with the interface hierarchy. Or by "type" are you > referring to a new interface which combines a couple interfaces? > > Note that I think it is quite valid to state that interfaces must always > be a tree, although I don't see any reason to avoid multiple-inheritance. So if we allow multiple inheritance then they will not always be a tree, right? In my working draft, "Class" is a sub-interface of both "Interface" and "Callable". > >... > > Interface expression language: > > ============================== > > These are normally called "type declarators". I would suggest using > standard terminology here. We aren't dealing with types. We are dealing with interfaces. And we aren't dealing with declarators, but with expressions. These expressions can be used in contexts other than type declarations. "Neither Holy, nor Roman nor an Empire" - Voltaire > Just use the "dotted_name" construct here -- that is well-defined by the > Python grammar already. It also provides for things like > "os.path.SomeInterface". The construct is fine for the grammar but it doesn't describe the semantics. > Note that interfaces do *not* have to occur in a PyDL module. Leave the > spec open for a combined syntax -- we shouldn't be required to declare all > interfaces in separate files. Interfaces in a Python file are automatically extracted and are thus available in a PyDL module. > In other words, the lengths do not have to be equal. A precondition is > that all union typedecls must be "flattened" to remove other unions. The > resulting, flattened list must then follow your equivalency algorithm. Okay, I'll see what I can do about it. > > 3. parameterize a interface: > > > > Array( Int, 50 ) > > Array( length=50, elements=Int ) > > > > Note that the arguments can be either interface expressions or simple > > Python expressions. A "simple" Python expression is an expression that > > does not involve a function call or variable reference. > > I disagree with the notion of expressions for the parameter values. I > think our only form of parameterization is with typedecl objects. The type > checker is only going to be dealing with type information -- expression > values as part of an interface don't make sense at compile time. Parameters would be made available to implementing classes at runtime. I see a lot of virtue in numeric bounds, string prefixes and so forth: typedecl colors as Enum(elements=["Red","Green","Blue"]) > I agree. The return type should also be optional. Note that we can't allow > just a name (and no type), as that would be ambiguous with just a type > name. I like the explicitness of requireing a return type and I harbor hopes that Python will one day distinguish between NO return type and something that happens to be able to return None. > > Note that at this point in time, every Python callable returns > > something, even if it is None. The return value can be named, > > merely as documentation: > > > > def( Arg1 as Int , ** as {String: Int}) - > ReturnCode as Int > > Ack! ... no, I do not think we should allow names in there. Return values > are never named and would never be used. Parameters actually have names, > which the values are bound to. A return value name also introduces a minor > problem in the grammar (is the name a name for the return value or a type > name?). How is the issue different in the return code versus in parameters? I think that this is a very useful features for IDEs and other documentation and has zero cost. > >... > > 2. Basic attribute interface declarations: > > > > decl myint as Int # basic > > decl intarr as Array( Int, 50 ) # parameterized > > decl intarr2 as Array( size = 40, elements = Int ) # using keyword > > syntax > > "as" does make sense in this context, but I'd use colons for consistency. The inconsistency is very minor and I am somewhat uncomfortable with appearing to begin a suite. I doubt that programmers would even notice the inconsistency. > > So this is allowed: > > > > class (_X,_Y) spam( A, B ): > > decl someInstanceMember as _X > > decl someOtherMember as Array( _X, 50 ) > > > > .... > > You haven't introduced this syntax before. Is this a class definition? Er, yes, but I don't have that syntax in the language anymore. Just change "class" to "interface" > > These are NOT allowed: > > > > decl someModuleMember(_X) as Array( _X, 50 ) > > Reason: modules are not parameterizable. No, the reason was stated before. Because *attributes* like someModuleMember cannot be declared to need incomplete interfaces. Only interfaces can be incomplete. > However: I think modules should be able to conform to an interface. And > since an interface can be parameterized, then this means that a module can > be parameterized. This is analogous to parameterizing a class. > > > class (_Y) spam( A, B ): > > decl someInstanceMember(_X) as Array( _X, 50 ) > > > > Because that would allow you to create a "spam" without getting around > > to saying what _X is for that spam's someInstanceMember. That would > > disallow static type checking. > > Agreed. The _X must occur in the class declaration statement. No, that's another typo. Here's another example and it comes back to the fact that attributes cannot be incomplete: interface (_Y) spam( A, B ): decl someInstanceMember(_Y) as Array( _Y, 50 ) > >... > > It is possible to allow _X to vary to some extent but still require it > > to always be a Number: > > > > decl Add(_X as Number) as def( a as _X, b as _X )-> _X > > Note that this implies the concept of hierarchy among the interfaces. Yes, that was also implied by the graph that started with Any. That is now explicit:

An interface may be derived from (or based upon) another interface called the base interface using Python inheritance syntax. Objects directly supporting a derived interface are said to indirectly support the base interface and its base interfaces all of the way up to the most basic interface, Any. > Note that you will then have to define a > rule for whether "decl x as Int" is the "same" as "decl x as Number". For > conformance, is the first too specific, or is it just a more concrete form > of the latter? (but still allowed) Well, in general there is no problem specifying a base versus derived interface. Your choice of specificity. The "Int" is a special case because it is also an implementation specific interface derived from > > The syntax for a class definition is identical to that for a function > > with the keyword "def" replaced by "class". What we are really > > defining is the constructor. The signature of the created object can > > be described in an interface declaration. > > Ick. We don't need anything special for this. The constructor is given by > the __init__ that occurs in the interface. > > > decl TreeNode(_X) as class( > > a as _X, > > Right as TreeNode( _X ) or None, > > Left as TreeNode( _X ) or None ) > > -> ParentClasses, Interfaces > > This would be: > > class (_X) TreeNode(ParentClasses): > __interfaces__ = Interfaces > def __init__(self, a: _X, > Right: TreeNode(_X) or None, > Left: TreeNode(_X) or None): > ... I don't want to introduce a new kind of interface declaration for classes. You should use ordinary interface declarations. There is no need for a new kind of "class-y" interface declaration and it will likely be abused so that more code is implementation specific than it needs to be. > If you're just trying to create the notion of a factory, then "def" is > appropriate: > > decl TreeNode(_X): def(a: _X, > Right: TreeNode(_X) or None, > Left: TreeNode(_X) or None) \ > -> (ParentClasses or Interfaces) No, we need ot differentiate functions from classes because classes can be subclassed. Otherwise there is no difference. That's why all we do is change the keyword. > These should be assignments and use a unary operator. The operator is much > more flexible: > > print_typedecl_object(typedef Int or String) > > Can't do that with a typedef or decl *statement*. You can't do it in one line, but you can do it. It is of debatable utility anyhow. The vast majority of the time you want to introspect interface objects that are *in use* not hard-coded ones. Introspection is really a secondary consideration anyhow. > Also note that your BoundedInt example is a *runtime* parameterization. > The type checker can't do anything about: > > decl x: PositiveInt > x = -1 That's true. I don't see that as a problem. > But we *can* check something like this: > > def foo(x: NegativeInt): > ... > decl y: PositiveInt > y = 5 > foo(y) I'm curious how you would see your type inferencer knowing whether to inference j=6 as "int", "PositiveInt" or "NegativeInt" Anyhow, you still don't prevent: decl y: NegativeInt y=5 foo(y) > But this latter case is more along the lines of naming a particular type > of Int. The syntax could very well be something like: > > decl PositiveInt: subtype Int > decl NegativeInt: subtype Int No need for the keyword "subtype". Two different int typedefs should both be usable as ints (I think), but not as each other. > The type-checker would know that PositiveInt is related somehow to Int > (and it would have to issue warnings when mixed). Argh. More warnings. I do not view it within our purview to require implementations to issue warnings. We define something as legal or as illegal. Anything else is between the implementor and the user. > It would also view > PositiveInt and NegativeInt as different (thereby creating the capability > for the warning in the foo(y) example above). > > Anyhow... as I mentioned above, we should only be allowing typedecl > parameters. We can't type-check value-based parameters. Not at compile time, but we can provide them to the implementation object which can check them at runtime. > If you want to introduce a type name for a runtime type-enforcement (a > valid concept! such as your PositiveInt thing), then we should allow full > expressions and other kinds of fun in the parameter expressions (since the > runtime type should be createable with anything it wants; we've already > given up all hope on it). But then we get into problems trying to > distinguish between a type declarator and an expression. For example: > > MyType = typedef ParamType(0, Int or String) > > In this example, the first is an expression, but the second should be a > type declarator. Figuring out which is which is tricky for the parser. Well maybe we need to just make the expression syntax unifiable with Python syntax. I am not comfortable to say that a type expression will never occur unadorned in Python code or vice versa. > I disagree that they will always be extracted into a separate PyDL file. > As an optimization: sure, we could do this. Effectively like caching a > module's bytecodes in a .pyc file. But I don't think you should codify > that here. It makes the specification easier to write and understand. > >... > > "typesafe": > > =========== > > In addition to decl and typedecl the keyword "typesafe" can be used to > > indicate that a function or method uses types in such a way that each > > operation can be checked at compile time and demonstrated not to call > > any function or operation with the wrong types. > > What about the problem of non-existence? How "safe" is "typesafe"? And how > is this different from regular type checking? It is how you *turn on* regular type checking at compile time. > >... > > An interface checker's job is to ensure that methods that claim to be > > typesafe actually are. It must report and refuse to compile modules > > that misuse the keyword and may not refuse to compile modules that do > > not. > > That last sentence is awkward. Can you rephrase/split/etc? Yes. > Class definitions also have the parameterization syntax change: > > class (_X) Foo(Super): > decl node: _X > ... No, that was a bug in the spec. Classes are declared just like functions except for the "class" keyword. They behave just like functions except that they can be subclassed. > Class and modules should also have a syntax for specifying the > interface(s) they conform to. I think that that is extracted from the class declaration's return type automatically. We will have to invent something for modules. "moddecl" or something. > Don't you mean "list of string", or should you drop the brackets? Thanks, fixed. > > __conforms__ : def (obj: Any ) -> boolean > > Just call it "conforms". There is no need to "hide" this method since the > interface does not expose interface members as its *own* members. It is __conforms__ for the same reason that __repr__, __cmp__ and __init__ are hidden: it is actually invoked through the magic "isa" syntax, not directly. I would be amenable to getting rid of the underscores Python-wide, but in any case I want to be consistent. > I think that you would want a version that just checks an objects > __interfaces__ attribute (quick), and a different method that does an > exhaustive check of the object's apparent interface against the specified > interface. I'll add that as an issue. > > Experimental syntax: > > ==================== > > > > There is a backwards compatible syntax for embedding declarations in a > > Python 1.5x file: > > > > "decl","myint as Integer" > > Just use a single string. The parse tree actually gets even uglier if you > put that comma in there :-). We can pull the "decl" out just as easily if > it is the first part of a "decl myint: Integer". I don't want to get confused with docstrings. > Why pull them out? Leave them in the file and use them there. No need to > replicate the stuff to somewhere else (and then try to deal with the > resulting synchronization issues). Because we always pull them out to a distinguishable file name, there are no more synchronization issues than with Python .pyc files. > I'd say the temporary syntax could be another function call: > > assert_type(x, "Int or String") Okay. > > The runtime should not allow an assignment or function call to violate > > the declarations in the PyDL file. In an "optimized speed mode" those > > checks would be disabled. In non-optimized mode, these assignments > > would generate an IncompatibleAssignmentError. > > This is a difficult requirement for the runtime. I would suggest moving > this to a V2 requirement. I don't see that our version numbers and Python's version numbers have to coincide. If it takes two years for Python to live up to all of the rules of our spec, so be it. > > The runtime should not allow a read from an unassigned attribute. It > > should raise NotAssignedError if it detects this at runtime instead of > > at compile time. > > Huh? We already have a definition for this. It raises NameError or > AttributeError. Please don't redefine this behavior. It is a different error. NameError's and AttributeError's should be eliminated through static type checking (where it is used religiously). NotAssignedError is not always statically detectable. Implementing this is also easy since objects have access to their interface objects. > >... > > Idea: The Undefined Object: > > =========================== > > You haven't addressed any of my concerns with this object. Even though > you've listed it under the "future" section, I think you're still going to > have some serious [implementation] problems with this concept. It is about as difficult to handle as pervasive assignment checks and both will probably be part of a written-from-scratch Python. Paul Prescod From paul@prescod.net Tue Jan 4 04:41:55 2000 From: paul@prescod.net (Paul Prescod) Date: Mon, 03 Jan 2000 23:41:55 -0500 Subject: [Types-sig] feedback: PyDL RFC 0.4 References: Message-ID: <38717A13.12D84874@prescod.net> Tony Lownds wrote: > > Why not allow a default type to a parameterized interface? > > class (T:=Any) Set: > ... > > decl f as def(x: Set) > decl g as def(x: Set(Int)) Why ":=" and not just "="? > Paul, is it your intention that... > a tuple is a Sequence? > a list can be a Record? > > I hope so, I think these syntactic shortcuts will be good if so. Yes, and yes. You could also define your own record types. > Can one declare a Mapping of more than one pair of types? > ie, {T1: T2, T3: T4} Is this close enough: {T1 or T3: T2 or T4} I think that's closer to the mathematical concepts of domain and range. > You dont show a way to specify default arguments. How about: > > def(Arg1: String, Arg2: String = optional) -> Int Here are other ideas: def(Arg1: String, Arg2: String?) -> Int def(Arg1: String, [Arg2: String]) -> Int Paul Prescod From gstein@lyra.org Tue Jan 4 10:09:08 2000 From: gstein@lyra.org (Greg Stein) Date: Tue, 4 Jan 2000 02:09:08 -0800 (PST) Subject: [Types-sig] feedback: PyDL RFC 0.4 In-Reply-To: <38717A13.12D84874@prescod.net> Message-ID: On Mon, 3 Jan 2000, Paul Prescod wrote: > Tony Lownds wrote: >... > > Can one declare a Mapping of more than one pair of types? > > ie, {T1: T2, T3: T4} > > Is this close enough: > > {T1 or T3: T2 or T4} That is different from Tony's request. Your typedecl would allow for {T1:T4} and {T3:T2} items to exist in the dictionary. Tony says each item must be T1:T2 or T3:T4. Note that this is also different from: {T1:T2} or {T3:T4}. In this case, the entire dictionary must be one or the other -- you could not mix items in the same dict. > I think that's closer to the mathematical concepts of domain and range. Tony asked about a particular form... whether it has a corresponding mathematical concept is beside the point :-) Personally: I find it is getting a bit too detailed. I would say that the (necessary) expressibility ends with Paul's example. > > You dont show a way to specify default arguments. How about: > > > > def(Arg1: String, Arg2: String = optional) -> Int > > Here are other ideas: > > def(Arg1: String, Arg2: String?) -> Int This one isn't bad, but the "?" doesn't bind very strongly to the fact that Arg2 is the optional item (not the typedecl). If you wrote it as: def(Arg1: String, Arg2?: String) -> Int then you'd have something a little closer to the intent. It just doesn't feel very nice, though... it is a bit too different from a normal definition: def foo(Arg1: String, Arg2 = 'hi': String) -> Int: ... [ note that I point out in my proposal that the grammar could easily manage either: "arg=expr:type" or "arg:type=expr". I prefer the former, for reasons listed in the proposal doc. ] > def(Arg1: String, [Arg2: String]) -> Int This would introduce parsing ambiguity with a list type declarator. Of the three forms, I think Tony's is probably best so far. I think there may be a better syntax, though... somebody just has to think of it :-) Cheers, -g -- Greg Stein, http://www.lyra.org/ From gstein@lyra.org Tue Jan 4 12:50:08 2000 From: gstein@lyra.org (Greg Stein) Date: Tue, 4 Jan 2000 04:50:08 -0800 (PST) Subject: [Types-sig] feedback, round 2 (LONG) (Re: PyDL RFC 0.4) In-Reply-To: <38717991.6B2E0B58@prescod.net> Message-ID: On Mon, 3 Jan 2000, Paul Prescod wrote: > Thanks for your feedback. I will need a lot more before we are done this > thing! And thanx for the detailed response :-) ... let's see if we can get another round in place here... > Greg Stein wrote: > > Wouldn't these be called "abstract interfaces" or "parameterized > > intefaces"? That seems to be a more standard terminology. > > I don't like "parameterized interface" because Sequence(Int) *is* > parameterized. I need to distinguish Sequence(_X) from Sequence(Int). > > Abstract is a little better, but we aren't dealing with abstract classes > as they are known in C++ or Java. Yah. I didn't really like either of my suggestions, for similar reasons. However, I think we want to reserve the name "incomplete interface" for the recursive-type situation. Maybe you could use the terms "parameterized interface" for Sequence(_X) and "bound interface) for Sequence(Int). Hrm. Maybe "unbound" and "bound". What do you think? > > > Typedefs allow us to give names to complete or incomplete interfaces > > > described by interface expressions. Typedefs are an interface > > > expression re-use mechanism. > > > > typedefs are also used to assign names to things like "Int or String". > > > > I don't see "Int" as an interface (even though it probably is in *theory*, > > it doesn't seem that way in layman usage). > > It makes the spec much easier to read and write if we think of them > uniformly as interfaces. Else we must constantly refer to "interfaces > and thingees like Int and String." Well... a name in a type declarator could refer to several objects: a class, an interface, a type object, or a typedecl object. Each of these certainly has an associated interface, but they are not (all) interfaces. You're right no the Int thing. I withdraw, but will still point out that a typedef can refer to more things than interfaces. For example: a = typedef types.IntType or types.StringType b = typedef a or types.ListType In the former, "a" refers to a typedecl object which is the union of two type objects. The the latter, "b" refers to a union between a typedecl object and another type object. I would amend your statement to read: "typedefs allow us to give names to interface expressions." [ note that I will still argue they are type declarators ] Your language also implicitly/subtly seems to state that we will *generate* an (unnamed) interface as the result of an interface expression. For sanity's sake, I would ask that we avoid that notion... it will then add Yet Another Type of Interface to our already burgeoning set of complete, incomplete, bound, unbound, concrete, abstract, parameterized, or whatever you want to call them :-). At the basic level, we have interface objects and a type declarator will not automatically generate composite interface objects. You can argue it mathematically that this happens, but I've seen too many specs that fall into that trap and become hard to understand. Please just have one notion of interface objects and avoid generated/composite/synthesized interfaces. > > And I still don't understand the need to specify that *two* files exist. > > Why are we going to look for two files? Isn't one sufficient? > > One is where you put your hand-written declarations. The other is where > the interpreter dumps the declarations that it extracts from the Python > file. That way you can use all inline declarations, a separate file or > *both* with no danger of having your hard work overwritten. But this is my part of my point (also covered in other parts of that email, but I'll summarize here): I don't believe that we will necessarily create that second file. I think it is a spurious requirement to state that we will extract/replicate inline definitions out into another file. You rightfully point out somewhere that an implementation can take the inline stuff and cache it somewhere, but let's have the spec deal with only *two* inputs: the inline data and an interface file. If the implementation has cached the inline data somewhere... fine... but let's not spec that as another file to deal with. This also helps to reduce our problem set to: given an interface declaration inline and an interface declaration in an interface file, how do we merge them? [ note that a module may have multiple interfaces declared inline and each interface may occur in a different file, but on a per-interface basis, we only need to worry about two locations. ] Adding a second file means we must deal with (discuss) three inputs. We can point out the third is just a copy of the inline data, but then why should we bring that up in the spec? Just talk about inline and an interface file. And what will the rules be for an interface showing up in multiple places? For example, let's say that I define an interface three times in my module and twice in interface file A and another twice in interface file B. When I type check the whole bugger, I feed it the module and two interface files. Do we get an error, or do they resolve somehow? Are the interfaces in the interface file(s) always referred to by module.name, or do they jammed into the module namespace somehow? (this probably depends on the definition of the external tool's process) > > In the above example, we have three interface objects. One is available > > via the name "foo1" in the module-level namespace. One is available as > > "Bar.foo2" (via the class' namespace, and the class is in the module > > namespace). The third, foo2, is only available within the function Baz(). > > We're making a static type checking system. I don't see what runtime > definition of interfaces in a function scope buys other than confusion. > If we need to have interface decarlations in random contexts then we > should differentiate compile-time available ones with a "decl" keyword > prefix. The different interfaces are to deal with scoping of their usage. We scope classes and functions all the time in Python. Nothing says that we must always place them at the global level. Maybe I want a private interface to be used within my class. For example: class MyClass: interface Item: decl member a: Int decl member b: Int decl member _items: List{Item} ... I don't want to be forced to move that interface out to the global namespace. Nor do I want to worry about whether "Item" interferes with an Item defined elsewhere -- that name occurs within a "class" definition, so I expect it to be scope that way. You mention runtime definition. Your words :-). I'm talking about compile-time definition and scoping of the interfaces. Sure, there is also a runtime component that assigns an interface object to MyClass.Item, *but* that does not negate the scoping. [ and yah: if you track down my original comment, it started out with the basis of runtime objects being assigned to names in different namespaces. So sue me :-) ... the scoping is the basic requirement for wanting those names in the different namespaces. ] > > I do not believe there is a need to place the interfaces into a distinct > > namespace. I'd be happy to hear one (besides forward-refs, which can be > > handled by an incomplete interface definition). > > A static type checking system exists to precede and constrain dynamism, > not to expand it. Huh? I don't see how that answers the need to place them into a different namespace. It just sounds like some mumbo-jumbo. AFAIK, a static type checking system exists to check whether you've written your program correctly. [ A good compiler can use that to impute certain constraints and thereby optimize the program, but that is a side effect rather than the main, original purpose. ] Back to my supposition: I do not believe we need to place names and interface objects into a distinct namespace. I believe these can go into the namespace of the context where the interface is defined. And my question: what is the requirement that establishes the need for a new namespace? [ namespace in terms of resolving names; a dict of name:interface is just a dict. ] >... > > > Certain interfaces may have only one implementation. These "primitive" > > > types > > > are Int, Long, Float, String, UnboundMethods, BoundMethods, Module, > > > Function > > > and Null. Over time this list may get shorter as the Python > > > implementation is generalized to work mostly by interfaces. > > > > I don't understand what you're saying here. This paragraph doesn't seem to > > be relevant. > > It is crucial to the distinction between implementations and interfaces. > Certain types do not have such a distinction so you cannot just > implement the right attributes and expect it to "work". You cannot make > a new class that Python treats as an Integer. You cannot make a new > class that MFC treats as a window handle. Ah. Understood. Your original statement didn't make this clear :-) > Here's what I say in my > current working version: > > > Sometimes there exists code that is only compatible with a single > > implementation of an interface. This is the case when the object's > > actual bit-pattern is more important than its interface. Examples > > include integers, window handles, C pointers and so forth. For this > > reason, every class is considered also an interface. Only instances of > > the class and its subclasses (if any) conform to the interface. These > > are called "implementation specific interfaces." The line about "bit-pattern" is superfluous. I think that should be omitted as it is distracting. I would also change the wording to: "Every class implies a specific interface. Only instances of this class and its subclasses (if any) conform to this implicit interface." The "for this reason" doesn't make sense. I don't see the connection between the preceeding sentences and the next sentence. Also, the "its subclasses" is a bit shaky. Consider: class Foo: def f1(self, x): ... class Bar(Foo): def f1(self): Foo.f1(self, 5) The subclass has a different signature for f1(). While it is pretty uncommon to use this pattern on random methods, it is *very* common for the __init__ method. The subclass is used to fill in specific parameters for the superclass constructor. Considering that the __init__ method forms part of the implicit interface, then the Bar subclass does not conform to that interface. I understand where you're going with the paragraph now, and it is definitely needed. However, I think the topic needs a bit more fleshing out. > > > Note: The Python interface graph may not always be a tree. For > > > instance there might someday be a type that is both a mapping and a > > > sequence. > > > > In the above statement, you're mixing up implementations (which can use > > disjoint interfaces) with the interface hierarchy. Or by "type" are you > > referring to a new interface which combines a couple interfaces? > > > > Note that I think it is quite valid to state that interfaces must always > > be a tree, although I don't see any reason to avoid multiple-inheritance. > > So if we allow multiple inheritance then they will not always be a tree, > right? Correct. Multiple inheritance creates a DAG rather than a tree. If you aren't careful, you might actually get cycles: decl incomplete class Foo class Bar(Foo): ... class Foo(Bar): ... Oops! :-) Actually, a simple rule can toss this out: don't allow incomplete classes for one of the base classes. And to my comment: I was pointing out that we could specify a *rule* that it must be a tree. But I don't see any reason to do that... multiple inheritance on interfaces seems fine to me. [ the Java designers say otherwise, though ] > In my working draft, "Class" is a sub-interface of both "Interface" and > "Callable". Then you have a DAG rather than a tree :-) I'd simply ask that you clear up that text a bit. When I saw "type", I thought of the builtin Python types. It looks like you meant "interface". > > >... > > > Interface expression language: > > > ============================== > > > > These are normally called "type declarators". I would suggest using > > standard terminology here. > > We aren't dealing with types. We are dealing with interfaces. And we > aren't dealing with declarators, but with expressions. These expressions > can be used in contexts other than type declarations. In this context, "type" and "interface" are the same. We are not referring to the builtin types. The "expressions" you refer to are exactly analogous to what C calls a "type declarator" -- a thing which declares a specific type. int [] That is a type declarator. It refers to an array of ints. We are using the same semantic concept, but with Pythonic syntax and base types. We are also constructing a composite [type declarator] from basic pieces. This composite specifies a type that an object may have. Yes, you could say that the composite specifies the interface that the object may conform to. But: a type declarator can be more than an interface, as I mentioned above: types.IntType or types.StringType In this case, the declarator refers to actual PyType objects, not interface specifications. The compile- and run- time checks would look at the ob->ob_type field ... not the __interfaces__ attribute (which might not even exist!). If we go back to you "implementation specific intefaces" concept... you're really referring to a PyType object rather than an interface spec. >... > > Just use the "dotted_name" construct here -- that is well-defined by the > > Python grammar already. It also provides for things like > > "os.path.SomeInterface". > > The construct is fine for the grammar but it doesn't describe the > semantics. Then add some text... geez. The point is that the text you provided does not seem to allow deeply nested interface references. It also recreates some of the dotted_name grammar construct, but isn't as clear about it. > > Note that interfaces do *not* have to occur in a PyDL module. Leave the > > spec open for a combined syntax -- we shouldn't be required to declare all > > interfaces in separate files. > > Interfaces in a Python file are automatically extracted and are thus > available in a PyDL module. I disagree, as mentioned above. If an implementation happens to cache inline data, then that is its business. Let's leave it out of the spec. >... > > > 3. parameterize a interface: > > > > > > Array( Int, 50 ) > > > Array( length=50, elements=Int ) > > > > > > Note that the arguments can be either interface expressions or simple > > > Python expressions. A "simple" Python expression is an expression that > > > does not involve a function call or variable reference. > > > > I disagree with the notion of expressions for the parameter values. I > > think our only form of parameterization is with typedecl objects. The type > > checker is only going to be dealing with type information -- expression > > values as part of an interface don't make sense at compile time. > > Parameters would be made available to implementing classes at runtime. I > see a lot of virtue in numeric bounds, string prefixes and so forth: > > typedecl colors as Enum(elements=["Red","Green","Blue"]) Woah, Nelly! Hrm... let me find a reference here... http://www.python.org/pipermail/types-sig/1999-December/000776.html Basically, the message points out that runtime-parameterization is a huge problem. Python is just not (yet) set up to handle runtime parameterization. Constructors? Yes. If your Enum() example referred *only* to a construction of an Enum instance which is then used in some way to perform type checks... then yah. I could see something like this. But let's be very clear. Using runtime parameterization with something like this is out of the question: class (_X) Foo: ... Also, I'd like to note that your example could be rewritten as: # construct enum instance RGB_Enum = Enum(elements=["Red","Green","Blue"]) # create a typedef (I'm assuming you meant "typedef ...") typedef colors as RGB_Enum But the typedef line isn't really necessary since it is a simple alias for the class instance: RGB_Enum = Enum(...) colors = RGB_Enum or: colors = Enum(...) Presuming that later you will be doing something like: def foo(rgb: colors)->Int: ... or: rgb = getColors(image) ! colors It is *very* interesting to note that your example here has effectively duplicated some of the compile-time vs. run-time distinctions from one of my previous emails: http://www.python.org/pipermail/types-sig/2000-January/001095.html In that email, I stated that the type-checker could not do much with a construct such as: a = doSomething() def foo(x: a)->None: ... And that it would defer to a runtime check. (and if the option flags were set properly, it would issue a "suspicious construct" warning or something). If you see the equivalence to value-expression-based parameterization and my two referenced postings, then I believe we may be getting somewhere! :-) In summary: torch value-expression parameterization -- only allow type-expression parameterization. Your value-expression-based example is actually a "type-checker-punted-to-runtime" example, which I believe we should be able to handle as such. > > I agree. The return type should also be optional. Note that we can't allow > > just a name (and no type), as that would be ambiguous with just a type > > name. > > I like the explicitness of requireing a return type and I harbor hopes > that Python will one day distinguish between NO return type and > something that happens to be able to return None. No problem. Please mark this as an issue in the RFC. There is at least one vote for requiring the return type, and one for making it optional. Let's see some other discussion/input. Personally, I want to be able to mark one or two function parameters as having types, but not worry about some other params or the return types. i.e. when I'm retrofitting some code > > > Note that at this point in time, every Python callable returns > > > something, even if it is None. The return value can be named, > > > merely as documentation: > > > > > > def( Arg1 as Int , ** as {String: Int}) - > ReturnCode as Int > > > > Ack! ... no, I do not think we should allow names in there. Return values > > are never named and would never be used. Parameters actually have names, > > which the values are bound to. A return value name also introduces a minor > > problem in the grammar (is the name a name for the return value or a type > > name?). > > How is the issue different in the return code versus in parameters? Parameters use those names in binding values to names in the local namespace. The return code name is never bound anywhere. It is totally superfluous. Barely even handy as documentation. "ReturnCode" doesn't say much :-) If you want doc, then use a docstring (rather than introducing even more syntax). And: as I mentioned above: there are definite grammar issues with allowing a NAME right there. The parser can't tell whether the NAME is a return value name or the first part of a type declarator. Tony has already pointed out this problem in some of my current syntax change proposals. I've gotta go jigger up some ugly grammar stuff to solve that. I would hope that we can keep the ugly grammar stuff minimized. > I think that this is a very useful features for IDEs and other > documentation and has zero cost. If you feel strongly, then okay... mark down another issue in the RFC to collect input on this :-) > > >... > > > 2. Basic attribute interface declarations: > > > > > > decl myint as Int # basic > > > decl intarr as Array( Int, 50 ) # parameterized > > > decl intarr2 as Array( size = 40, elements = Int ) # using keyword > > > syntax > > > > "as" does make sense in this context, but I'd use colons for consistency. > > The inconsistency is very minor and I am somewhat uncomfortable with > appearing to begin a suite. I doubt that programmers would even notice > the inconsistency. I noticed it :-) If colons are used in function parameters, then we should use colons in the declarations. > > > So this is allowed: > > > > > > class (_X,_Y) spam( A, B ): > > > decl someInstanceMember as _X > > > decl someOtherMember as Array( _X, 50 ) > > > > > > .... > > > > You haven't introduced this syntax before. Is this a class definition? > > Er, yes, but I don't have that syntax in the language anymore. Just > change "class" to "interface" We should. I want to parameterize classes. Don't force me to extract an interface from my class definition -- I want an implicit, parameterized interface derived from my class definition and its inline declarations. There are issues with the syntax for parameterizing classes and instances (as mentioned in another email tonite), but I *do* want to parameterize both. The basic premise is that I will use classes as *both* an interface definition and an implementation. To that end, I want all the features of a standard inteface definition to be available through my class definition. > > > These are NOT allowed: > > > > > > decl someModuleMember(_X) as Array( _X, 50 ) > > > > Reason: modules are not parameterizable. > > No, the reason was stated before. Because *attributes* like > someModuleMember cannot be declared to need incomplete interfaces. Only > interfaces can be incomplete. All right. I see your distinction. Not sure what I was thinking :-) I might have glossed over the first (_X) and just saw the second. Module attributes should be able to have that second _X within their interface definitions. >... > > > class (_Y) spam( A, B ): > > > decl someInstanceMember(_X) as Array( _X, 50 ) > > > > > > Because that would allow you to create a "spam" without getting around > > > to saying what _X is for that spam's someInstanceMember. That would > > > disallow static type checking. > > > > Agreed. The _X must occur in the class declaration statement. > > No, that's another typo. Here's another example and it comes back to the > fact that attributes cannot be incomplete: > > interface (_Y) spam( A, B ): > decl someInstanceMember(_Y) as Array( _Y, 50 ) Gotcha. I completely agree. If you rewrite to: interface (_Y) spam( A, B ): decl someInstanceMember as Array( _Y, 50 ) then it is legal. [ with the caveats that a class is also parameterizable, I think we need to use different syntax for parameterization, and that value expressions should not occur in a parameter binding. ] >... > > Note that you will then have to define a > > rule for whether "decl x as Int" is the "same" as "decl x as Number". For > > conformance, is the first too specific, or is it just a more concrete form > > of the latter? (but still allowed) > > Well, in general there is no problem specifying a base versus derived > interface. Your choice of specificity. The "Int" is a special case > because it is also an implementation specific interface derived from Ignore the Int thing. Let's say I have the following: interface Foo: ... interface Bar(Foo): ... The question is now: how are these viewed from a conformance standpoint? -- Are List{Foo} and List{Bar} the same? Probably not. -- If I have a List{Bar}, can I pass it to something that asks for a List{Foo}? Should be able to. But what if that function inserts a Foo into my List{Bar}? Oops! -- If I have a List{Foo}, can I pass it to something that asks for a List{Bar}? Probably not. The target wants Bar objects, which Foo objects definitely are not. These rules will need to be defined at some point [for function parameter passing/binding]. This sub-email-thread started with a parameterization of the form: decl Add(_X as Number) as def(... The binding of a type to _X would follow similar rules to those defined above for function parameters. >... > I don't want to introduce a new kind of interface declaration for > classes. You should use ordinary interface declarations. There is no > need for a new kind of "class-y" interface declaration and it will > likely be abused so that more code is implementation specific than it > needs to be. Yes, but I do. I feel very strongly against having to separate an interface definition out of my implementation. If I have to say: interface TreeNodeInterface{_X}: def __init__(self, a: _X, Right: TreeNodeInterface{_X} or None, Left: TreeNodeInterface{_X} or None) class TreeNode{_X}: __interfaces__ = TreeNodeInterface def __init__(self, a: _X, Right: TreeNodeInterface{_X} or None, Left: TreeNodeInterface{_X} or None): my_code() then I'll scream. I'll scream even louder if that interface declaration has to go into a separate file. The maintenance would be painful. And note that I have no idea how to remove the _X parameterization from the class definition. I need/want to include the declarations in the __init__ function and need the specific parameterization of TreeNodeInterface. I think... I'm not even so sure how this thing would work in this case. Basically: I need to be able to parameterize a class definition. And don't dare to tell me that the implied interface of that class is "abused so that more code is implementation specific than it needs to be." As the application developer, this is my choice. I do not want dual-path maintenance, and the binding of interface to implementation is entirely appropriate in my application. > > If you're just trying to create the notion of a factory, then "def" is > > appropriate: > > > > decl TreeNode(_X): def(a: _X, > > Right: TreeNode(_X) or None, > > Left: TreeNode(_X) or None) \ > > -> (ParentClasses or Interfaces) > > No, we need ot differentiate functions from classes because classes can > be subclassed. Otherwise there is no difference. That's why all we do is > change the keyword. You shouldn't be able to subclass a class without an actual definition being present. Otherwise, you could end up with loops (as I mentioned above). Therefore, you have factories, or you have classes. But you don't declare a class as a constructor function -- you define the whole thing. [ or declare it "incomplete" per my suggestion above... not even the constructor is known in that case... not until definition time. ] > > These should be assignments and use a unary operator. The operator is much > > more flexible: > > > > print_typedecl_object(typedef Int or String) > > > > Can't do that with a typedef or decl *statement*. > > You can't do it in one line, but you can do it. It is of debatable > utility anyhow. The vast majority of the time you want to introspect > interface objects that are *in use* not hard-coded ones. Introspection > is really a secondary consideration anyhow. My example was extremely simple. You cannot discount the concept based on that. Here, I'll give you a better example: def some_kind_of_apply_thing(func, params): arg_tds = func.func_signature.values() td = arg_tds[0] for arg_td in arg_tds[1:]: td = typedef td or arg_td for param in params: if not td.check(param): raise ArgTypeError return apply(func, params) Is that better? The typedef operator is a *big* win in terms of clarity and utility. And as I've argued before, the type-checker can easily understand all the forms of: x = typedef some.typedecl.expression that you were proposing to allow in the "typedef" statement (e.g. only dotted names, parameterizations, or other simple constructions). > > Also note that your BoundedInt example is a *runtime* parameterization. > > The type checker can't do anything about: > > > > decl x: PositiveInt > > x = -1 > > That's true. I don't see that as a problem. It is a problem if you are shooting for *only* compile-time checking. I've been recommending the ability to punt to some runtime checks for a while, and it seems that you've been against them. It sounds like you're starting to allow for them now. The above construct is fine, then. I simply was trying to point out that you wouldn't be happy with it because we couldn't do anything with it at compile time. > > But we *can* check something like this: > > > > def foo(x: NegativeInt): > > ... > > decl y: PositiveInt > > y = 5 > > foo(y) > > I'm curious how you would see your type inferencer knowing whether to > inference > > j=6 > > as "int", "PositiveInt" or "NegativeInt" "Int". All integer constants are "Int". The compile-time checker has no way to determine the semantics of value-expression-based parameterized types such as your PositiveInt or NegativeInt. Since it can't know the semantics, then it can't classify the integer constant appropriately. > Anyhow, you still don't prevent: > > decl y: NegativeInt > y=5 > foo(y) Correct. (and your point is...?) I never figured we could prevent this because of the value-expression binding for NegativeInt causing the whole thing off to be punted off to runtime. [ and for other reasons, the value-expression binding can't be done ] > > But this latter case is more along the lines of naming a particular type > > of Int. The syntax could very well be something like: > > > > decl PositiveInt: subtype Int > > decl NegativeInt: subtype Int > > No need for the keyword "subtype". Two different int typedefs should > both be usable as ints (I think), but not as each other. Agreed. I was trying to shoot for a possible syntax that would provide the compile-time checker with that semantic. In the "subtype" example, the checker can know this now. decl a: PositiveInt decl b: NegativeInt a = 5 ! PositiveInt b = -1 | NegativeInt func_taking_ints(a, b) # success a = b # failure The checker knows that PositiveInt and NegativeInt are subtypes of Int and can therefore be bound to the params of func_taking_ints(). But it knows they are different types, so it raises an error on the assignment. > > The type-checker would know that PositiveInt is related somehow to Int > > (and it would have to issue warnings when mixed). > > Argh. More warnings. I do not view it within our purview to require > implementations to issue warnings. We define something as legal or as > illegal. Anything else is between the implementor and the user. I have *never* recommended to issue runtime warnings. I was referring to a compile-time warning about mixing the two types. e.g. between the checker and the implementor. And the checker will most certainly issue warnings. > > It would also view > > PositiveInt and NegativeInt as different (thereby creating the capability > > for the warning in the foo(y) example above). > > > > Anyhow... as I mentioned above, we should only be allowing typedecl > > parameters. We can't type-check value-based parameters. > > Not at compile time, but we can provide them to the implementation > object which can check them at runtime. While true, we cannot truly parameterize classes at runtime. We can only create instances [for use with runtime type-checking] with different constructor values. In that sense, we *still* do not have value-based parameterization. > > If you want to introduce a type name for a runtime type-enforcement (a > > valid concept! such as your PositiveInt thing), then we should allow full > > expressions and other kinds of fun in the parameter expressions (since the > > runtime type should be createable with anything it wants; we've already > > given up all hope on it). But then we get into problems trying to > > distinguish between a type declarator and an expression. For example: > > > > MyType = typedef ParamType(0, Int or String) > > > > In this example, the first is an expression, but the second should be a > > type declarator. Figuring out which is which is tricky for the parser. > > Well maybe we need to just make the expression syntax unifiable with > Python syntax. I am not comfortable to say that a type expression will > never occur unadorned in Python code or vice versa. I posit that type declarator syntax can never be mixed directly with standard expression syntax. [Int] has very different meanings based on whether you're talking about a type declarator or an expression. The typedef operator is *intended* to allow this mixing. The RHS of the operator is a type declarator, it produces a value [for use in a Python expression]. >... > > > "typesafe": > > > =========== > > > In addition to decl and typedecl the keyword "typesafe" can be used to > > > indicate that a function or method uses types in such a way that each > > > operation can be checked at compile time and demonstrated not to call > > > any function or operation with the wrong types. > > > > What about the problem of non-existence? How "safe" is "typesafe"? And how > > is this different from regular type checking? > > It is how you *turn on* regular type checking at compile time. Isn't it also enabled by the presence of declarations? I also believe that the checker won't be integrated directly into the compiler. But that doesn't negate the question: how safe is "typesafe" and how does that differ from regular type checking (e.g. when I put a declaration on a function parameter). >... > > Class definitions also have the parameterization syntax change: > > > > class (_X) Foo(Super): > > decl node: _X > > ... > > No, that was a bug in the spec. Classes are declared just like functions > except for the "class" keyword. They behave just like functions except > that they can be subclassed. I disagree. Explained above. > > Class and modules should also have a syntax for specifying the > > interface(s) they conform to. > > I think that that is extracted from the class declaration's return type > automatically. We will have to invent something for modules. "moddecl" > or something. Classes are not always declared. They may simply be defined. (and certainly, the module is plain-old-defined) I have proposed that we define the interfaces conformed-to by assigning them to the __interfaces__ attribute of the class and module. >... > > > __conforms__ : def (obj: Any ) -> boolean > > > > Just call it "conforms". There is no need to "hide" this method since the > > interface does not expose interface members as its *own* members. > > It is __conforms__ for the same reason that __repr__, __cmp__ and > __init__ are hidden: it is actually invoked through the magic "isa" > syntax, not directly. I would be amenable to getting rid of the > underscores Python-wide, but in any case I want to be consistent. Different cases. class Foo: def __init__(self): self.a = 5 def __str__(self): return str(self.a) The __str__ needs to have the underscores to avoid conflicts with "a". An interface object does not expose the defined-attributes as its own attributes. Hence, there is no need for conflict avoidance. conforms() will be quite sufficient. Jim Fulton has some other methods defined for interface objects that may be interesting. >... > > > There is a backwards compatible syntax for embedding declarations in a > > > Python 1.5x file: > > > > > > "decl","myint as Integer" > > > > Just use a single string. The parse tree actually gets even uglier if you > > put that comma in there :-). We can pull the "decl" out just as easily if > > it is the first part of a "decl myint: Integer". > > I don't want to get confused with docstrings. Well... the checker certainly won't be confused. Humans? Nah. Really: go generate a parse tree for that comma-separated bugger sometime. Then try to fit that in with a mechanism for extracting the declaration. It will be a bitch times three. If/when I add the string-based declaration stuff to my check.py prototype, I'll be doing it as a single string. I truly don't want to deal with commas between a couple strings. I doubt any other person who goes to implement a prototype (or the real thing!) will want to either. > > Why pull them out? Leave them in the file and use them there. No need to > > replicate the stuff to somewhere else (and then try to deal with the > > resulting synchronization issues). > > Because we always pull them out to a distinguishable file name, there > are no more synchronization issues than with Python .pyc files. When I say "why pull them out?" ... "Because we always pull them out" is not a helpful answer :-) *why* ? In any case, I've covered this ground above. I maintain that we leave them as inline data. If the implementation wants to do something under the covers, then fine... it has no affect whatsoever on the user. >... > > > The runtime should not allow an assignment or function call to violate > > > the declarations in the PyDL file. In an "optimized speed mode" those > > > checks would be disabled. In non-optimized mode, these assignments > > > would generate an IncompatibleAssignmentError. > > > > This is a difficult requirement for the runtime. I would suggest moving > > this to a V2 requirement. > > I don't see that our version numbers and Python's version numbers have > to coincide. If it takes two years for Python to live up to all of the > rules of our spec, so be it. But we also don't want to bite off more than we can chew. If we put in something about "the runtime disallows assignments of the wrong type" and create an *expectation* that this will be followed, then we will quickly find ourselves in a trap. This won't be implemented any time soon, so why discuss it or even pretend that we'll be getting around to it in the near future? Punt it to V2. Set people's expectations properly. > > > The runtime should not allow a read from an unassigned attribute. It > > > should raise NotAssignedError if it detects this at runtime instead of > > > at compile time. > > > > Huh? We already have a definition for this. It raises NameError or > > AttributeError. Please don't redefine this behavior. > > It is a different error. NameError's and AttributeError's should be > eliminated through static type checking (where it is used religiously). > NotAssignedError is not always statically detectable. It sounds like you're changing the error for change's sake. Why not leave it as NameError or AttributeError? Why introduce another name for an existing semantic? try: x = foo.bar except AttributeError: ... Are you saying that I have to go and change all these to use NotAssignedError? Why? If it is a direct replacement, then there is no new semantic and therefore no need to change the name. > Implementing this is also easy since objects have access to their > interface objects. I don't understand this. Implementation for "not assigned" is already done. It exists today in Python without the need for interface objects. Hrm. All right... let's clarify your language some: "If a name or attribute is not defined, then the checker may flag read accesses to these non-existent entities. If the name/attribute *is* defined, but a runtime access finds that it has not yet been assigned, then NotAssignedError is raised instead." Now your section makes sense. But I still don't buy it. Distinguishing between Name/AttributeError and NotAssignedError is probably of little value (IMO). At least: please mark the thing as an open issue. > > >... > > > Idea: The Undefined Object: > > > =========================== > > > > You haven't addressed any of my concerns with this object. Even though > > you've listed it under the "future" section, I think you're still going to > > have some serious [implementation] problems with this concept. > > It is about as difficult to handle as pervasive assignment checks and > both will probably be part of a written-from-scratch Python. It's worse than the assignments, I think. But hey: as long as that thing stays down in the "futures" section, then I'm not worried. And when the time comes, I won't be the volunteer to code it up :-) Cheers, -g -- Greg Stein, http://www.lyra.org/ From m.faassen@vet.uu.nl Tue Jan 4 13:26:51 2000 From: m.faassen@vet.uu.nl (Martijn Faassen) Date: Tue, 04 Jan 2000 14:26:51 +0100 Subject: [Types-sig] check.py (was: PyDL RFC 0.02) References: <000601bf528c$70a62920$a02d153f@tim> Message-ID: <3871F51B.6CAE9870@vet.uu.nl> Tim Peters wrote: > > [Paul Prescod] > > My syntax is mostly based on your {GregS's] web page. I switched > > "!" for "as" based on my belief that it isn't Pythonic to use > > random keyboard characters in ways that are not universally > > understood... > > FYI, in Common Lisp the name of this function is the delightful "the"; e.g., > > (the integer (somefunc i)) > > looks at the value returned by (somefunc i), passes it along if it's an > integer, else raises an error. > > and-some-people-say-lisp-is-unreadable-ly y'rs - tim Neat! def foo(a the Integer, b the String, c the Foo, d the Any) the Integer: e = d the Integer return e I could definitely live with that one. Doesn't have the semantic confusion that 'as' have and it seems readable. Regards, Martijn From tony@metanet.com Tue Jan 4 17:20:47 2000 From: tony@metanet.com (Tony Lownds) Date: Tue, 4 Jan 2000 09:20:47 -0800 (PST) Subject: [Types-sig] feedback: PyDL RFC 0.4 In-Reply-To: <38717A13.12D84874@prescod.net> Message-ID: On Mon, 3 Jan 2000, Paul Prescod wrote: > Tony Lownds wrote: > > > > Why not allow a default type to a parameterized interface? > > > > class (T:=Any) Set: > > ... > > > > decl f as def(x: Set) > > decl g as def(x: Set(Int)) > > Why ":=" and not just "="? > Oops, I didnt mean to introduce the colon. It means something in my mind, FWIW: Using a colon would narrow the types a type parameter can take: class (T: Number) Foo: ... Using an equals would specify a default type: class (T=Int) Foo: ... Using both is fine: class (T:Number = Int) Foo: ... So in my mind using both would specify a default type and narrow the types at the same time. class (T:=Int) Foo: ... which is the same as class (T:Int=Int) Foo: ... > > Paul, is it your intention that... > > a tuple is a Sequence? > > a list can be a Record? > > > > I hope so, I think these syntactic shortcuts will be good if so. > > Yes, and yes. You could also define your own record types. > I dont know if defining your own record types is a good idea. decl f as def() -> (Int, String, Float) decl a as Int, b as String, c as Float a,b,c = f() If f returned a user-defined Record, the type checker would allow it but the runtime would break. > > Can one declare a Mapping of more than one pair of types? > > ie, {T1: T2, T3: T4} > > Is this close enough: > > {T1 or T3: T2 or T4} > > I think that's closer to the mathematical concepts of domain and range. > Yeah, thats close enough. > > You dont show a way to specify default arguments. How about: > > > > def(Arg1: String, Arg2: String = optional) -> Int > > Here are other ideas: > > def(Arg1: String, Arg2: String?) -> Int > > def(Arg1: String, [Arg2: String]) -> Int > That last one wouldn't work with out parameter names: decl f as def(String, [String]) -> Int -Tony From paul@prescod.net Tue Jan 4 17:03:07 2000 From: paul@prescod.net (Paul Prescod) Date: Tue, 04 Jan 2000 12:03:07 -0500 Subject: [Types-sig] Re: syntax and compile/run -time References: Message-ID: <387227CB.E4F4F796@prescod.net> Greg Stein wrote: > > On Mon, 3 Jan 2000, Paul Prescod wrote: > > Greg Stein wrote: > > > Dividing the language into pieces is simply dividing it. I see no rational > > > reason for doing so, but many reasons to keep everything clean and > > > integrated. > > > > Dividing the language is inevitable. The question is whether to do it > > "up front" or subtly through cryptic error messages. > > You are presuming that the alternative to "up front" is "cryptic error > messages". Again: this is FUD. It is my opinion that if you try to pretend that things that are static are dynamic, then all you do is delay the shock of finding out that things don't work as you expected. > > I don't want the type-checker flagging valid code with warnings (unless > > you turn -Wall) and I don't want it silently ignoring code like the > > above because it doesn't understand it. > > I agree. I've been presuming that a number of warnings will only be > flagged with -Wall. Whether my explanation on what happens with your > example truly issues a warning or not... fine-tuning. Okay, then we can put aside warning messages. Code like this is just "valid": a=callSomeFunction() decl foo def(x: a)->None Code that calls foo is not statically type checkable. What is the point of using static type checking syntax if this cannot be static type checked? They is just as easy: a=callSomeFunction() def foo( x ): assert a in x.__interfaces__ I don't consider it intuitive to use static type checking syntax for something that must be evaluated at runtime. A static declaration that can't be checked statically just isn't useful. > Your syntax with a "typedef" statement attempts to avoid the problem by > restricting the syntax to an arbitrary subset of Python. My proposal > allows people to use similar, limited syntax *or* to use complex stuff for > complex, runtime operations. I don't restrict runtime manipulation of type objects. I just think that it is done through a reflection API similar to the traceback API or the parse tree API. > Fine. We can declare your example "illegal". But doesn't "this should be legal" fall naturally out of the syntax? I think that the whole point of the syntax is that types are just runtime objects and can be treated just as runtime objects. > I was simply trying to state > that it *can* be legal. Look at my words, quoted above. I was attempting > to move it to #3 -- runtime only. Even though we move it there, the > type-checker can tell that the user may have thought it fell into #1, so > it can issue a warning. > > But hey: this is what Guido calls, to paraphase, "appropriate error > messages." Some fine-tuning is all. I'm having trouble arguing here because we haven't even decided whether it is illegal or not. If it is illegal, then that is confusing because what's the point of treating type objects as ordinary Python objects with ordinary assignments if they are going to be restricted in some contexts? If it is legal, you said in the paragraph above that it is reasonable that the user is going to be confused about what is going on and how it works. Since it is legal, there will be no error message so the user is on her own unless she thinks to use "lint mode". > ==> "incomplete" classes and interfaces may be declared I've never pre-declared a function or class in Python. Don't you think that predeclaring interfaces is un-Pythonic? > decl incomplete class foo_private > decl incomplete class foo_object Ugh. > Oh, bunk. I was attempting to make it actually *work* for the user. In > your model, it would just punt. You are creating artificial limitations, > and you're doing it through a partitioned syntax. Neither is desirable. I see a partitioned syntax as a virtue. It is very analogous to the distinction between languages that feel that they have to make regular expressions "first class" integrated syntax and those that say: "regular expressions use a different model than everything else. They will be a sub-language with special syntax and semantics." > If somebody says: > > MyComplicatedType = doSomething() > > def foo(x: MyComplicatedType): > ... > > Why should we prevent them? Because declaring a static type check that cannot be statically evaluated is not a reasonable action, IMO. It will most often be an accident. It is not Pythonic to allow the dubious action and then depend on a "lint mode" to warn of the dubiousness. I say, let them do it, but tell them they > aren't going to get compile-time assistance with the construct. > > Keep the syntax clean. Provide flexibility. Let the developer know if they > have done something suspicious. "Do or do not do. There is no try." - Yoda "Legal or illegal. There is no suspicious." - Paul Can we put aside warnings? I think that we agree that they are not a solution. The only way to "let the developer know" when they have done something supicious is through error messages. That means that the construct is illegal. As soon as we make a few of these constructs illegal, we are back to essentially a sub-language proposal. If we do NOT make it illegal then we have to allow a compiler to silently accept it and the programmer is on his own. Paul Prescod From paul@prescod.net Tue Jan 4 18:42:25 2000 From: paul@prescod.net (Paul Prescod) Date: Tue, 04 Jan 2000 13:42:25 -0500 Subject: [Types-sig] OPINIONS: Warnings Message-ID: <38723F11.B5939730@prescod.net> This is an important issue that I would like a lot of feedback about. I will state my opinion and invite Greg and others to state theirs. It is my opinion that we should not, in our specification, flag any dubious constructs as being constructs that the compiler should warn about. Anything dubious should be merely disallowed. Warnings are for getting around language flaws. C++ has if( a=b ){...} and "lint" has to warn about it. Python just disallows the construct and the warning becomes superflurous. C++ banished a lot of error prone C-isms for exactly this reason. Unfortunately, many of those that were too pervasive (in code) to remove were left behind. Sociologically speaking, we all know that "suspect" constructs tend to get either deprecated to "defacto errors" ("you should NEVER, EVER, do that") or get ignored. So things tend towards defacto legality or illegality anyhow. IMO, we will never be able to totally get rid of warning and "lints" but we should acknowledge them as byproducts of imperfect designs created by imperfect humans. If we design any part of our system *expecting* a warning to be issued then we are knowingly designing in flaws. If a commonly used construct is likely to cause confusion then we should just ban it unless doing so has some other drastic consequence. Therefore we should banish warnings from our lexicon for the purposes of this discussion. Opinions? Paul Prescod From paul@prescod.net Tue Jan 4 18:55:41 2000 From: paul@prescod.net (Paul Prescod) Date: Tue, 04 Jan 2000 13:55:41 -0500 Subject: [Types-sig] feedback: PyDL RFC 0.4 References: Message-ID: <3872422D.FFF7942D@prescod.net> Greg Stein wrote: > > > Is this close enough: > > > > {T1 or T3: T2 or T4} > > That is different from Tony's request. Your typedecl would allow for > {T1:T4} and {T3:T2} items to exist in the dictionary. That's why I asked if it was close enough. :) > Of the three forms, I think Tony's is probably best so far. I think there > may be a better syntax, though... somebody just has to think of it :-) Okay, it's an issue. Paul Prescod From gstein@lyra.org Tue Jan 4 19:29:15 2000 From: gstein@lyra.org (Greg Stein) Date: Tue, 4 Jan 2000 11:29:15 -0800 (PST) Subject: [Types-sig] updated proposal (fwd) Message-ID: Missed on the CC: to the mailing list... forwarding with permission... ---------- Forwarded message ---------- Date: Mon, 3 Jan 2000 22:48:25 -0500 From: scott To: Greg Stein Subject: Re: [Types-sig] updated proposal On Mon, Jan 03, 2000 at 06:41:07PM -0800, Greg Stein wrote: > I've updated my proposal document at: > > http://www.lyra.org/greg/python/typesys/type-proposal.html > > > There have been some minor refinements to text, examples, and a small > syntax change to type declarator (had to add '(' typedecl ')' to deal with > some binding precedence issues). > > I've also added a big section on what I see as open issues. I've listed by > biased :-) thoughts along with my understanding of the counter-proposal. > > Comments are most welcome! glad to see you have so much time on your hands! some small comments: """ I propose the the use of the __interfaces__ attribute, although Jim Fulton's Scarecrow Proposal uses the __implements__ attribute, so that may be the preferred attribute. """ I like __interfaces__ better too. what operator to use as type-assert operator: I prefer 'isa' or 'asa'. If I still used emacs, I'd _hate_ to have to wait for python-mode.el to catch up to understanding the colons in type-assert operators, short of that, I prefer the colon, it looks familiar. In compile-time changes, you wrote: """ Check that the left-hand side of a type-assert operator may sometimes succeed. If it can never succeed, then an error is flagged. """ Is there any way (ie restrictions on variable name usage, etc) to make this more strict? I'd much prefer that it be able to check that the left hand side of a type assert operator must succeed and follow some reasonable guidelines to make it do so than have this whole category made so wisshy-washy at compile time. regarding a new namespace to place interface/type info into... I'd suggest leaving that open for the time being, or just defining that access is available at runtime via a function call such as: interfaces() -> [ all known interfaces in current namespace ] interfaces(x: Any) -> [ all known interfaces that x implements ] this would allow 'implementors' to put the darn info wherever they want and define a flexible enough interface to getting the interfaces to allow for variable underlying implementations. (sorry for mouthful...) Re: 'associating interfaces with a module': I don't see any reason to define interfaces so that modules can't have them: python is good at module-level interfaces, I use them more frequently than I've seen others do (don't like big files, no matter what editor or tags stuff I'm using). Re: """ I am unsure whether interfaces should include the notion of class versus instance attributes, or whether separate interfaces should be defined and used. If separate interfaces are to be used, then we would probably want a __class_interfaces__ attribute to list the interface(s) that the class object conforms to. """ Ouch, I like to KISS, an interface is an interface is an interface... if the definition of it isn't good enough, then we better make it better (ie make the notion of interface understand class vs instance variables because they're different and both contribute to an interface). There should be interfaces for all objects, and those we can't make right off the bat we should try to make sure we don't put off in such a way that they will become unduly tricky down the road. I think this is a key concept we should strive for... in the long run, all built-in objects should have a set of interfaces, and those interfaces should be defined in terms of atomic (resolved) types wherever possible. re: """ The change to varargslist allows a lambda's arguments to have type declarators. I haven't thought this fully through, but is this entirely necessary? Do we need to actually type-check those? Lambdas are typically not used in cross-unit interfaces where the type checking is most needed. """ I say trash 'm if there really a problem grammatically. We can still do the same with nested defs, which are generally more efficient given the choice of doing either anyway. re: """ One other possible syntax change would be to introduce an interface keyword. This would lexically replace the class keyword in Python code, and its body would be similar to a class body except that attributes could be declared but not assigned and functions could have doc strings, but no code body. I'm not sure that this buys us anything over just using a regular class and adhering to those rules. """ I like the interface keyword, even if it only happens inside a decl statement so it won't have to be a keyword: the reason is that I can see it being much simpler to name interfaces and classes the same thing in many cases where ``KlassInterface'' would just jumble up the namespace with two different names. Basically, I'd like the ability to follow C-style declarations here where you can name interfaces the same thing as the actual thing, and I think that should extend to classes. It would help to tighten the bond between the frequent 1-1 relationship between interfaces and classes. sorry for the random order of things, and wish I could put more time into this, but that's just not going happen just yet. scott > > Cheers, > -g > > -- > Greg Stein, http://www.lyra.org/ > > > _______________________________________________ > Types-SIG mailing list > Types-SIG@python.org > http://www.python.org/mailman/listinfo/types-sig From paul@prescod.net Tue Jan 4 20:24:12 2000 From: paul@prescod.net (Paul Prescod) Date: Tue, 04 Jan 2000 15:24:12 -0500 Subject: [Types-sig] Typesafe Message-ID: <387256EC.C8DAD36F@prescod.net> Let's see if we can figure out this typesafe thing once and for all because we've half-covered it a dozen times: > > It is how you *turn on* regular type checking at compile time. > > Isn't it also enabled by the presence of declarations? No, I don't think so. Case 1: decl getprop as def( foo: URL, prop: String ) -> String def getprop( foo, prop ): return WebDAVGetPropertyThatIKnowIsAString( foo, prop ) I declared the return type for *clients* of this function, not because I want to check the body of the function itself. The function itself is trivial. I know it works. I'm a Python programmer and I don't aways need a compiler holding my hand as if I were an infant. That isn't strictly speaking typesafe and a Java programmer would do something like this: decl getprop as def( foo: URL, prop: String ) -> String def getprop( foo, prop ): return WebDAVGetPropertyThatIKnowIsAString( foo, prop ) isa String But that's why I don't use Java. Sometimes I just know what a thing is and I don't feel like telling the compiler. This is going to especially be the case when I am calling reams and reams of non-type-safe code that I happen to trust (e.g. Zope!). I don't want to cast every method call. But then other times I write hundreds of lines of mission critical code and I want a little help to guarantee that it does what I think: typesafe def getprop( foo, prop ): 10,000 lines return WebDAVGetPropertyThatIKnowIsAString( foo, prop ) isa String Case 2: decl getprop as def( foo: URL, prop: String ) -> String def getprop( foo, prop ): j = someUntypeSafeFunction() + someOtherUntypesafeFunction() return "abc" Does this code violate any of its declarations? No. Is it typesafe? No. Is it legal? Yes. Python allows you to add random variables if you think you know what you are doing. But this is *not* legal: typesafe def getprop( foo, prop ): j = someFunction() + someOtherFunction() return "abc" Now I've asked the compiler to pretend I am an incompetent child (or a Java programmer ) and tell me when I have done anything dangerous. --- Type declarations are you helping the compiler because it is too stupid to figure out what is going on. typesafe declarations are about the compile helping you because you might not be smart enough to figure out what's going on. Despite my tounge in cheek jabs at Java programmers, I think that typesafety would be the norm in my own code, but it should not be the default because Python is not Java. And one virtue of "typesafe" is that as a side effect it guarantees that you stick to a Python subset that is directly compilable into (fairly) idiomatic (and thus efficient) Java or C++. If you leave off the typesafe then there may be some implicit runtime type checks going on that will sap your speed. Paul Prescod From paul@prescod.net Tue Jan 4 20:42:49 2000 From: paul@prescod.net (Paul Prescod) Date: Tue, 04 Jan 2000 15:42:49 -0500 Subject: [Types-sig] Re: feedback, round 2 (LONG) (Re: PyDL RFC 0.4) References: Message-ID: <38725B49.831FE79C@prescod.net> Greg Stein wrote: > > .... > > Yah. I didn't really like either of my suggestions, for similar reasons. > However, I think we want to reserve the name "incomplete interface" for > the recursive-type situation. Well, we don't know if we'll need that yet. :) Okay, since there is no easy resolution, we'll call this an issue. > Maybe you could use the terms "parameterized interface" for Sequence(_X) > and "bound interface) for Sequence(Int). Hrm. Maybe "unbound" and "bound". > What do you think? I think that Python already has a concept of unbound. If we were willing to give up symmetry we could talk about "concrete interfaces" and "interface factory interfaces." The thing is that we still want these things to be interfaces, not just interface factories, because they have interface declarations and most of the other trappings of interfaces. We might still revert to interface instances and interface factories. > > It makes the spec much easier to read and write if we think of them > > uniformly as interfaces. Else we must constantly refer to "interfaces > > and thingees like Int and String." > > Well... a name in a type declarator could refer to several objects: a > class, an interface, a type object, or a typedecl object. Each of these > certainly has an associated interface, but they are not (all) interfaces. In my terminology (and inheritance hierarchy) they all are. Typedecls *create* interace objects (and do nothing else!). Classes and type objects are automatically interfaces just as side effects. > Your language also implicitly/subtly seems to state that we will > *generate* an (unnamed) interface as the result of an interface > expression. For sanity's sake, I would ask that we avoid that notion... it > will then add Yet Another Type of Interface to our already burgeoning set > of complete, incomplete, bound, unbound, concrete, abstract, > parameterized, or whatever you want to call them :-). No, it isn't a new type of interface. It's just an unnamed interface. 1+3 isn't a new kind of number, it's just a new way to get a number. > At the basic level, > we have interface objects and a type declarator will not automatically > generate composite interface objects. You can argue it mathematically that > this happens, but I've seen too many specs that fall into that trap and > become hard to understand. Please just have one notion of interface > objects and avoid generated/composite/synthesized interfaces. We need some term for the things that are generated and they meet all qualifications of interfaces so interface seems appropriate. > Adding a second file means we must deal with (discuss) three inputs. We > can point out the third is just a copy of the inline data, but then why > should we bring that up in the spec? Just talk about inline and an > interface file. We'll need to mark this as an issue. > And what will the rules be for an interface showing up in multiple places? > For example, let's say that I define an interface three times in my module > and twice in interface file A and another twice in interface file B. When > I type check the whole bugger, I feed it the module and two interface > files. Do we get an error, or do they resolve somehow? I think that there are at most two interface files per module, the .pi and the .gpi. It is an open issue whether we should allow multiple declarations. At this point I think that we should just disallow it until experience shows that we need to be more flexible. > Are the interfaces in the interface file(s) always referred to by > module.name, or do they jammed into the module namespace somehow? (this > probably depends on the definition of the external tool's process) They are available as part of the module namespace somehow. > Maybe I want a private interface to be used within my class. I think that that should be out of scope at this point. I want the grammar to be non-context specific so that a "stupid" tool can extract the declarations easily. > And my question: what is the requirement that establishes the need for a > new namespace? The requirement is that some processing must occur before Python execution begins therefore by definition we cannot use any Python namespace. *at runtime* we could combine our static namespace with a Python namespace but before runtime we have no option. > The "for this reason" doesn't make sense. I don't see the connection > between the preceeding sentences and the next sentence. Okay, I'll try to clean that up. > Also, the "its subclasses" is a bit shaky. Perhaps we just have to disallow non-conforming subclasses for classes that are used as interfaces. Then the trick would be in knowing when a cass is used as an interface. > And to my comment: I was pointing out that we could specify a *rule* that > it must be a tree. But I don't see any reason to do that... multiple > inheritance on interfaces seems fine to me. Agreed. > [ the Java designers say otherwise, though ] Disagree. Mutiple inheritance of interfaces is legal in Java. Look what our friend Bill Tutt has to say on the topic. > Multiple subtyping is the ability to subtype from multiple classes > at the same time. In Java, the programmer may subtype as many times > as he wishes, as long as he does so via the interface mechanism. http://www.elj.com/eiffel/feature/inheritance/mi/review/#s5 > > In my working draft, "Class" is a sub-interface of both "Interface" and > > "Callable". > > Then you have a DAG rather than a tree :-) That's what I said! :) > I'd simply ask that you clear up that text a bit. When I saw "type", I > thought of the builtin Python types. It looks like you meant "interface". Exactly right and now fixed. > In this context, "type" and "interface" are the same. We are not referring > to the builtin types. The "expressions" you refer to are exactly analogous > to what C calls a "type declarator" -- a thing which declares a specific > type. > > int [] > > That is a type declarator. It refers to an array of ints. We are using the > same semantic concept, but with Pythonic syntax and base types. Okay, but C revolves around a concept of types. Python (arguably) is supposed to revolve around a concept of interfaces. "I don't care if you hand me a file thing, as long as it BEHAVES like a file thing." Our static type checker is really a static interface checker. For the rare case when we really care about types, we'll use the implied, implementation-specific interface. > But: a type declarator can be more than an interface, as I mentioned > above: > > types.IntType or types.StringType > > In this case, the declarator refers to actual PyType objects, not > interface specifications. The compile- and run- time checks would look at > the ob->ob_type field ... not the __interfaces__ attribute (which might > not even exist!). I think that we should just presume that the attribute exists. It gets too messy when we start dealing with special cases. > Woah, Nelly! > > Hrm... let me find a reference here... > > http://www.python.org/pipermail/types-sig/1999-December/000776.html Okay, let's move it to future issues. As an aside, though, I wouldn't handle this through on the fly subtyping. Rather I would make type-parameters available to the implementation as attributes or parameters to a method or something. Then it becomes entirely the implementor's responsibiity to use the arguments as she sees fit. > If you see the equivalence to value-expression-based parameterization and > my two referenced postings, then I believe we may be getting somewhere! > :-) I see some equivalence but also a lot of difference. I am talking about the static type system providing some input to the runtime system. You are talking about trying to integrate the two. > Personally, I want to be able to mark one or two function parameters as > having types, but not worry about some other params or the return types. > i.e. when I'm retrofitting some code I would say to just use "Any". I think Tony pointed out that we run into grammatical concerns if a "bare name" can be both an argument name or a type name. > > > Ack! ... no, I do not think we should allow names in there. Return values > > > are never named and would never be used. Parameters actually have names, > > > which the values are bound to. A return value name also introduces a minor > > > problem in the grammar (is the name a name for the return value or a type > > > name?). > > > > How is the issue different in the return code versus in parameters? > > Parameters use those names in binding values to names in the local > namespace. I'm asking how the grammatical issue is different. > The return code name is never bound anywhere. It is totally superfluous. > Barely even handy as documentation. "ReturnCode" doesn't say much :-) If > you want doc, then use a docstring (rather than introducing even more > syntax). Well, if I do database.append( 'a', 'b', 'c'), it is very relevant whether the integer returned is SuccessCode or RecordSetLength. > > I think that this is a very useful features for IDEs and other > > documentation and has zero cost. > > If you feel strongly, then okay... mark down another issue in the RFC to > collect input on this :-) Issued. > > The inconsistency is very minor and I am somewhat uncomfortable with > > appearing to begin a suite. I doubt that programmers would even notice > > the inconsistency. > > I noticed it :-) > > If colons are used in function parameters, then we should use colons in > the declarations. Issue. > We should. I want to parameterize classes. Don't force me to extract an > interface from my class definition -- I want an implicit, parameterized > interface derived from my class definition and its inline declarations. I see this as: "Future Directions: At some point in the future, PyDL files will likely be generated from source code using a combination of declarations within Python code and some sorts of interface deduction and inferencing based on various kinds of assignment." Note that I have also provided no syntax for specifying the types of functions in the function declarations. You make a separate declaration with the types. > The basic premise is that I will use classes as *both* an interface > definition and an implementation. To that end, I want all the features of > a standard inteface definition to be available through my class > definition. But an interface is an interface. A class is a type. A type can be used as an "implementation specific interface" but this is a bad habit that leads to weirdness like functions that work on file objects but blow up on FTP download streams. I agree that there should someday be a syntax for extracting an implementation *independent* interface from a class as a convenience but then we need to get into issues of "public/private" and so forth. After all, it will almost never be the case that a class's interface will be 100% identical to its implementation signature. > Ignore the Int thing. Let's say I have the following: > > interface Foo: > ... > interface Bar(Foo): > ... > > The question is now: how are these viewed from a conformance standpoint? > > -- Are List{Foo} and List{Bar} the same? Probably not. Agree. BTW, why are you using curly braces and not parens? > -- If I have a List{Bar}, can I pass it to something that asks for a > List{Foo}? Should be able to. But what if that function inserts a Foo > into my List{Bar}? Oops! Interesting. Here's how I see it: If List is NOT readonly, then there is an implicit __setitem__ method. Our problem is that the __setitem__ is not propery contravariant (it isn't more explicit than its base interface's). We can detect this and report it in the same way that we would for "ordinary" methods. (i'm strongly in favor of treating __ methods as "ordinarily" as possible...they just happen to have an alternate syntax) But if the list is immutable, then it doesn't have __setitem__ and the problem goes away. Immutability rears its ugly head again. > If I have to say: > > interface TreeNodeInterface{_X}: > def __init__(self, a: _X, > Right: TreeNodeInterface{_X} or None, > Left: TreeNodeInterface{_X} or None) > > class TreeNode{_X}: > __interfaces__ = TreeNodeInterface > def __init__(self, a: _X, > Right: TreeNodeInterface{_X} or None, > Left: TreeNodeInterface{_X} or None): > my_code() > > then I'll scream. I'll scream even louder if that interface declaration > has to go into a separate file. The maintenance would be painful. No, you say: interface TreeNodeInterface{_X}: decl leftchild: TreeNodeInterface{_X} or None decl rightchild: TreeNodeInterface{_X} or None decl TreeNode as class(self, a: _X, Right: TreeNodeInterface{_X} or None, Left: TreeNodeInterface{_X} or None) class TreeNode: def __init__(self, a, Right, Left): self.leftchild=Left self.rightchild=Right my_code() (mine is longer than yours but it fills in some details that you elided, so look carefully) The amount of duplication is actually quite small and the class definition is exactly as it exists today, which will comfort many type-uncomfortable Python programmers. We can add the __interfaces__ magically. We don't need to re-define the types. I am changing Python syntax as little as possible until our design is verified through lots and lots of real, type-safe code. Yes, someday I see something like this as a shortcut: public class TreeNode: public decl leftchild: TreeNodeInterface{_X} or None public decl rightchild: TreeNodeInterface{_X} or None public def __init__(self, a: _X, Right: ... , Left: ...): self.leftchild=Left self.rightchild=Right my_code() The implementation independent interface is implied by the keywords "public". > Basically: I need to be able to parameterize a class definition. And don't > dare to tell me that the implied interface of that class is "abused so > that more code is implementation specific than it needs to be." As the > application developer, this is my choice. I do not want dual-path > maintenance, and the binding of interface to implementation is entirely > appropriate in my application. Are you telling me that you often create classes where every single attribute is part of the signature? > > No, we need ot differentiate functions from classes because classes can > > be subclassed. Otherwise there is no difference. That's why all we do is > > change the keyword. > > You shouldn't be able to subclass a class without an actual definition > being present. Otherwise, you could end up with loops (as I mentioned > above). Loops are easily detected at compile time. > Is that better? The typedef operator is a *big* win in terms of clarity > and utility. A big win for a small percentage of the population. If we are going to integrate these things then I would rather just make the interface expression syntax compatible with Python expression syntax. I am not really comfortable with having a single line of Python code where "or" means two radically different things. > It is a problem if you are shooting for *only* compile-time checking. I've > been recommending the ability to punt to some runtime checks for a while, > and it seems that you've been against them. It sounds like you're starting > to allow for them now. I am not against runtime checks. But the rules of what checks are done at runtime and what checks are done at compile time have to be no-brainers. We should not get into a case where one of these is a compile time check and the other a runtime check: interface B: pass interface C: pass a = B or C decl j as def( arg: a ) a = typedef B or C decl j as def( arg: a ) > "Int". All integer constants are "Int". The compile-time checker has no > way to determine the semantics of value-expression-based parameterized > types such as your PositiveInt or NegativeInt. Since it can't know the > semantics, then it can't classify the integer constant appropriately. Can you see that this is a more general problem with inferencing? a = open( "foo" ) Is "a" statically restricted to a) file objects b) file-*like* objects c) read-only file objects If I assign to something fifteen levels down the inheritance tree, how does the inferencer know which level I *really* meant. And what about this: if something(): a = open_readable_file( "foo" ) else: a = open_read_write_file( "foo" ) Is "a" statically restricted to a) file objects b) file-*like* objects c) readable file objects union writable file objects > > > The type-checker would know that PositiveInt is related somehow to Int > > > (and it would have to issue warnings when mixed). > > > > Argh. More warnings. I do not view it within our purview to require > > implementations to issue warnings. We define something as legal or as > > illegal. Anything else is between the implementor and the user. > > I have *never* recommended to issue runtime warnings. I was referring to a > compile-time warning about mixing the two types. e.g. between the checker > and the implementor. Separately addressed. I propose we stop talking about warnings. > I posit that type declarator syntax can never be mixed directly with > standard expression syntax. [Int] has very different meanings based on > whether you're talking about a type declarator or an expression. Okay, but Sequence( Int ) does not. It returns an interface. If we stop overloading syntax (painful but possible) then we can avoid future problems. One mildly ugly way to handle it: a = (![Int] !or ![String]) or 5 > Classes are not always declared. They may simply be defined. (and > certainly, the module is plain-old-defined) I have proposed that we define > the interfaces conformed-to by assigning them to the __interfaces__ > attribute of the class and module. Back into our old debate: __interfaces__ = someFunctionThatUsesCOMToUnpickleInterfacesFromDB() > Different cases. > > class Foo: > def __init__(self): > self.a = 5 > def __str__(self): > return str(self.a) > > The __str__ needs to have the underscores to avoid conflicts with "a". I see no problem with Python just declaring a method called "str" on the Any interface. Programmers would just choose not to conflict with it just as they choose not to conflict with any other name on a base class. But Guido decided not to do that so I'm trying to follow his pattern. I see *your* point but from my point of view the decision to use __ was always relatively arbitrary and as far as I can see the only differentiator between __ functions and ordinary ones is that __ ones are covers for some magical syntax. > > I don't want to get confused with docstrings. > > Well... the checker certainly won't be confused. Why not? Are we going to disallow certain strings at the beiginning of docstrings now? > Really: go generate a parse tree for that comma-separated bugger sometime. > Then try to fit that in with a mechanism for extracting the declaration. > It will be a bitch times three. I don't see the problem. You are scanning for top-level tuples where the first item is a string. > > > Why pull them out? Leave them in the file and use them there. No need to > > > replicate the stuff to somewhere else (and then try to deal with the > > > resulting synchronization issues). > > > > Because we always pull them out to a distinguishable file name, there > > are no more synchronization issues than with Python .pyc files. > > When I say "why pull them out?" ... "Because we always pull them out" is > not a helpful answer :-) Parse that as: "the reason there are no synchronization issues is..." > This won't be implemented any time soon, so why discuss it or even pretend > that we'll be getting around to it in the near future? Punt it to V2. Set > people's expectations properly. We are here to decide what is legal and what is illegal. This is as illegal as it gets: decl foo as int foo="abc" Do you disagree? > At least: please mark the thing as an open issue. Okay. Paul Prescod From skip@mojam.com (Skip Montanaro) Tue Jan 4 21:33:30 2000 From: skip@mojam.com (Skip Montanaro) (Skip Montanaro) Date: Tue, 4 Jan 2000 15:33:30 -0600 (CST) Subject: [Types-sig] check.py (was: PyDL RFC 0.02) In-Reply-To: <3871F51B.6CAE9870@vet.uu.nl> References: <000601bf528c$70a62920$a02d153f@tim> <3871F51B.6CAE9870@vet.uu.nl> Message-ID: <14450.26410.693253.653903@beluga.mojam.com> Martijn> def foo(a the Integer, b the String, c the Foo, d the Any) the Integer: Martijn> e = d the Integer Martijn> return e Martijn> I could definitely live with that one. Doesn't have the Martijn> semantic confusion that 'as' have and it seems readable. I have to toss out the barb that when the most frequently used token in my Python code is "the" or "as" it will be time to find another language... (Someone please reassure me by stating emphatically that all this extra stuff will never be required!) I'm-blinkin'-not-winkin'-ly y'rs, Skip From jpe@arachne.org Tue Jan 4 21:32:30 2000 From: jpe@arachne.org (John Ehresman) Date: Tue, 4 Jan 2000 16:32:30 -0500 (EST) Subject: [Types-sig] Typesafe In-Reply-To: <387256EC.C8DAD36F@prescod.net> Message-ID: What is the definition of typesafe and what is the definition of isa (or "!" or whatever)? I ask because I thought typesafe meant no code in the function could raise a type-related exception (TypeError or AttributeError), but I don't think the following meets that criteria: > On Tue, 4 Jan 2000, Paul Prescod wrote: > But then other times I write hundreds of lines of mission critical code > and I want a little help to guarantee that it does what I think: > > typesafe def getprop( foo, prop ): > 10,000 lines > return WebDAVGetPropertyThatIKnowIsAString( foo, prop ) isa String Am I missing something? If my definition of typesafe is not correct, can someone define it in terms of what exceptions can be thrown at runtime? Also, at least one version of the RFC defined the type assertion operator as raising a subtype of AssertionError if the type of the value is incorrect; does this imply that the exception will only be raised in debug mode, like the AssertionError is? I think you would want it to apply in non-debug mode also so that optimized code can safely generated. John From paul@prescod.net Tue Jan 4 22:09:22 2000 From: paul@prescod.net (Paul Prescod) Date: Tue, 04 Jan 2000 17:09:22 -0500 Subject: [Types-sig] Typesafe References: Message-ID: <38726F92.E1FEB75D@prescod.net> John Ehresman wrote: > > What is the definition of typesafe and what is the definition of isa (or > "!" or whatever)? I ask because I thought typesafe meant no code in the > function could raise a type-related exception (TypeError or > AttributeError), I think that there is a different in kind between an error raised by an explicit check and one that is likely to be just an accident. Languages like Java and C++ actually disallow the latter. So I would raise TypeAssertionError which is unrelated (in terms of the hierarchy) from TypeError. > Also, at least one version of the RFC defined the type assertion operator > as raising a subtype of AssertionError if the type of the value is > incorrect; does this imply that the exception will only be raised in debug > mode, like the AssertionError is? I think you would want it to apply in > non-debug mode also so that optimized code can safely generated. Worth debating. Is code that fails an assertion "safe"? I claim no. It is in error but you just told the compiler that for optimization reasons you don't care. The same would go for type errors. Paul Prescod From jpe@arachne.org Tue Jan 4 22:05:52 2000 From: jpe@arachne.org (John Ehresman) Date: Tue, 4 Jan 2000 17:05:52 -0500 (EST) Subject: [Types-sig] Re: syntax and compile/run -time In-Reply-To: <387227CB.E4F4F796@prescod.net> Message-ID: Something to think about in terms of compile / runtime is that not all modules will be compiled at the same time. Consider a distributed object system which creates proxy objects on the fly so someone can connect to a remote object and make calls on it without having a .py or other file that corresponds to the exact interface of the remote object. It would be nice if the following worked: interface HasName: decl name as def() -> String def PrintName(o as HasName) -> None: print o.name() o = ConnectToRemoteObjectThatIThinkImplementsHasName() PrintName(o) Obviously this is a simplistic example, but I think what would be important in a more complex example is to test whether a object supports a given interface before performing operations on it. Both CORBA and COM have facilities for doing this sort of thing, but it would be nice to have a common, native way of doing this. Actually, this brings up the question of some sort of runtime support because the check to see if an object supports a given interface is often delayed until some piece of code asks for it because the check requires the name of the interface and (maybe) a remote procedure call. It would also be nice to be able to examine the interface of an object through the native reflection API. I don't know if these considerations requre that type objects be constructable at runtime because python source could be generated and the compiled on the fly, but I think they serve as examples of why the type system should be extensible at runtime. John From jpe@arachne.org Tue Jan 4 22:34:33 2000 From: jpe@arachne.org (John Ehresman) Date: Tue, 4 Jan 2000 17:34:33 -0500 (EST) Subject: [Types-sig] Typesafe In-Reply-To: <38726F92.E1FEB75D@prescod.net> Message-ID: On Tue, 4 Jan 2000, Paul Prescod wrote: > John Ehresman wrote: > > > > What is the definition of typesafe and what is the definition of isa (or > > "!" or whatever)? I ask because I thought typesafe meant no code in the > > function could raise a type-related exception (TypeError or > > AttributeError), > > I think that there is a different in kind between an error raised by an > explicit check and one that is likely to be just an accident. Languages > like Java and C++ actually disallow the latter. So I would raise > TypeAssertionError which is unrelated (in terms of the hierarchy) from > TypeError. Is the definition then that no TypeAssertionError can be raised in the function except by a raise or isa test statement? What about sub functions? I think that simply saying that no TypeAssertionError may be emitted from the function may be a better definition because it's simpler to explain and I guess I don't see why you would want a TypeAssertionError coming from a typesafe function. I think on should be able to raise and catch exceptions within the function body, but not be able to let them escape from it. > > Also, at least one version of the RFC defined the type assertion operator > > as raising a subtype of AssertionError if the type of the value is > > incorrect; does this imply that the exception will only be raised in debug > > mode, like the AssertionError is? I think you would want it to apply in > > non-debug mode also so that optimized code can safely generated. > > Worth debating. Is code that fails an assertion "safe"? I claim no. It > is in error but you just told the compiler that for optimization reasons > you don't care. The same would go for type errors. I guess I was thinking of writing code that used "isa" to test whether a value conformed to an interface just as I sometimes use isinstance now. But regardless of whether isa or isinstance should be used, consider a compiler that generated C code to be compiled and was able to use C int variables for Python Integer under certain conditions and the following: def f(): i = IntValue() # I know this returns an Integer, but it's not declared # so the compiler doesn't know that i isa Integer while i < 5: DoSomething() i = i + 1 It could not simply omit the integer check. Maybe in this case, it would need to do a type check anyway to convert between implementation objects anyway, but then what should it do if the check fails if isa is defined as not doing anything in the optimized case? I guess I lean toward consistant well-defined behavior on the debug/non-debug modes, even if it means giving up some performance. John From paul@prescod.net Wed Jan 5 03:01:45 2000 From: paul@prescod.net (Paul Prescod) Date: Tue, 04 Jan 2000 22:01:45 -0500 Subject: [Types-sig] Re: syntax and compile/run -time References: Message-ID: <3872B419.13164EA4@prescod.net> John Ehresman wrote: > > ... > It would also be nice to be able to examine the interface of an object > through the native reflection API. I don't know if these considerations > requre that type objects be constructable at runtime because python source > could be generated and the compiled on the fly, but I think they serve as > examples of why the type system should be extensible at runtime. We all agree that interfaces should be constructible at runtime. What we disagree on is whether runtime constructed interfaces can be used in the same syntactic contexts as statically constructed interfaces. a = GoFetchMeAnInteface() decl foo as def( arg: a ) -> String interface B( a ): ... ...and so forth. I claim that the syntax should restrict us to clearly separating the runtime from the compile time. Paul Prescod From gstein@lyra.org Wed Jan 5 03:17:22 2000 From: gstein@lyra.org (Greg Stein) Date: Tue, 4 Jan 2000 19:17:22 -0800 (PST) Subject: [Types-sig] Re: syntax and compile/run -time In-Reply-To: <3872B419.13164EA4@prescod.net> Message-ID: On Tue, 4 Jan 2000, Paul Prescod wrote: > John Ehresman wrote: > > ... > > It would also be nice to be able to examine the interface of an object > > through the native reflection API. I don't know if these considerations > > requre that type objects be constructable at runtime because python source > > could be generated and the compiled on the fly, but I think they serve as > > examples of why the type system should be extensible at runtime. > > We all agree that interfaces should be constructible at runtime. What we > disagree on is whether runtime constructed interfaces can be used in the > same syntactic contexts as statically constructed interfaces. > > a = GoFetchMeAnInteface() > decl foo as def( arg: a ) -> String > > interface B( a ): > ... > > ...and so forth. I claim that the syntax should restrict us to clearly > separating the runtime from the compile time. Maybe it goes with saying, but I'm on the opposite fence from Paul :-) I don't think we should restrict the runtime from using any [new] Python syntax. The above code should be completely legal and valid, but it would not be compile-time checkable. Cheers, -g -- Greg Stein, http://www.lyra.org/ From scott@chronis.pobox.com Wed Jan 5 03:42:40 2000 From: scott@chronis.pobox.com (scott) Date: Tue, 4 Jan 2000 22:42:40 -0500 Subject: [Types-sig] Re: syntax and compile/run -time In-Reply-To: References: <3872B419.13164EA4@prescod.net> Message-ID: <20000104224240.A98858@chronis.pobox.com> On Tue, Jan 04, 2000 at 07:17:22PM -0800, Greg Stein wrote: > On Tue, 4 Jan 2000, Paul Prescod wrote: > > John Ehresman wrote: > > > ... > > > It would also be nice to be able to examine the interface of an object > > > through the native reflection API. I don't know if these considerations > > > requre that type objects be constructable at runtime because python source > > > could be generated and the compiled on the fly, but I think they serve as > > > examples of why the type system should be extensible at runtime. > > > > We all agree that interfaces should be constructible at runtime. What we > > disagree on is whether runtime constructed interfaces can be used in the > > same syntactic contexts as statically constructed interfaces. > > > > a = GoFetchMeAnInteface() > > decl foo as def( arg: a ) -> String > > > > interface B( a ): > > ... > > > > ...and so forth. I claim that the syntax should restrict us to clearly > > separating the runtime from the compile time. > > Maybe it goes with saying, but I'm on the opposite fence from Paul :-) > > I don't think we should restrict the runtime from using any [new] Python > syntax. The above code should be completely legal and valid, but it would > not be compile-time checkable. one thing that bothers me about this is that the line 'decl foo as ...' may be somewhere completely different than the 'a = ...' line, and so it becomes unclear what is meant by the programmer. one possible solution to that clarity is something like this: a = GoFetchMEAnInterface() decl foo as def( arg: a ) -> String is an error (at compile time) , but a = GoFetchMEAnInterface() decl foo as runtime def( arg: a ) -> String is ok at compile time, but may or may not cause an error at runtime. I'm not against keeping the same basic syntax, but I do believe it should be made clear *in the decl statement* that it is not intended to be part of the compile-time stuff. If the programmer indeed intends that (and they really might), they need to be told about it in a way that is more obvious than a warning. scott scott From gstein@lyra.org Wed Jan 5 04:26:35 2000 From: gstein@lyra.org (Greg Stein) Date: Tue, 4 Jan 2000 20:26:35 -0800 (PST) Subject: [Types-sig] updated proposal (fwd) In-Reply-To: Message-ID: Great comments. Thanx for the feedback. I made a few more updates just recently (a new section on typedefs and added some notes to sections that need some updates). I've checked into CVS, so future changes can be more easily tracked -- you won't need to re-read the whole doc looking for changes :-) I'll send out CVS info later... Later is now... look at the bottom of the document. There is a link to an online viewer that can display changes. The URL again: http://www.lyra.org/greg/python/typesys/type-proposal.html Now to the comments: I've recorded some of your thoughts in the proposal. Take a look to ensure that I've properly specified your position. On Tue, 4 Jan 2000, scott wrote: >... > """ > I propose the the use of the __interfaces__ attribute, although Jim Fulton's > Scarecrow Proposal uses the __implements__ attribute, so that may be the > preferred attribute. > """ > > I like __interfaces__ better too. Noted in the document. > what operator to use as type-assert operator: > > I prefer 'isa' or 'asa'. If I still used emacs, I'd _hate_ to have to > wait for python-mode.el to catch up to understanding the colons in > type-assert operators, short of that, I prefer the colon, it looks > familiar. I don't think that anybody has suggested using the colon for the *operator*. It is grammatically ambiguous. Could you clarify your position, and I'll mark the tally in the proposal. Thx. > In compile-time changes, you wrote: > """ > Check that the left-hand side of a type-assert operator may sometimes > succeed. If it can never succeed, then an error is flagged. > """ > > Is there any way (ie restrictions on variable name usage, etc) to make > this more strict? I'd much prefer that it be able to check that the left > hand side of a type assert operator must succeed and follow some > reasonable guidelines to make it do so than have this whole category > made so wisshy-washy at compile time. Given the following code: decl a: Int b = a ! String The compiler will know that it will always fail. So it can flag it. In the following: decl a: Any b = a ! String It has no idea. So it lets it pass, to be evaluated at runtime. The operator *is* a runtime operator. The reason it is important to the compile-time checker, however, is that the checker now knows that "b" is a String. It cannot be anything else (or the exception would prevent the assignment). I'm not sure this could be called "wishy washy". I definitely should document the operator better in the proposal. I've left myself a TODO for this. If there is still something wishy-washy about it, then please clarify so that I can address the issue. > regarding a new namespace to place interface/type info into... I'd > suggest leaving that open for the time being, or just defining that > access is available at runtime via a function call such as: > > interfaces() -> [ all known interfaces in current namespace ] > interfaces(x: Any) -> [ all known interfaces that x implements ] > > this would allow 'implementors' to put the darn info wherever they > want and define a flexible enough interface to getting the interfaces > to allow for variable underlying implementations. (sorry for > mouthful...) Interesting thought, but I feel confident that we don't need another namespace, so I don't really see a reason to make the above compromise. I've noted this idea in my document. > Re: 'associating interfaces with a module': I don't see any reason to > define interfaces so that modules can't have them: python is good at > module-level interfaces, I use them more frequently than I've seen > others do (don't like big files, no matter what editor or tags stuff > I'm using). I've recorded this as a tally. > Re: > """ > I am unsure whether interfaces should include the notion of class > versus instance attributes, or whether separate interfaces should be > defined and used. If > separate interfaces are to be used, then we would probably want a > __class_interfaces__ attribute to list the interface(s) that the class > object conforms to. > """ > > Ouch, I like to KISS, an interface is an interface is an interface... > if the definition of it isn't good enough, then we better make it > better (ie make the notion of interface understand class vs instance > variables because they're different and both contribute to an > interface). I'm inclined to agree. Noted in the doc. > There should be interfaces for all objects, and those we can't make > right off the bat we should try to make sure we don't put off in such > a way that they will become unduly tricky down the road. I think this > is a key concept we should strive for... in the long run, all built-in > objects should have a set of interfaces, and those interfaces should > be defined in terms of atomic (resolved) types wherever possible. Agreed. I've been thinking about this quite a bit, so that I can add more code to the type checker to properly deal with operators (e.g. addition). Let's say you have the following interface: interface Foo: decl member __add__: def(self:Foo, value:Foo)->Foo Class instances supporting the Foo interface are capable of being added. They *also* have an "__add__" attribute. Consider a builtin type: it can be added, but it does not have an "__add__" attribute. How do we resolve this? I believe we do it this way: interface Int: decl intrinsic __add__: def(self:Int, value:Int)->Int This tells the checker that Int understands addition, but it does NOT actually have an __add__ attribute. Thoughts? > re: > """ > The change to varargslist allows a lambda's arguments to have type > declarators. I haven't thought this fully through, but is this > entirely necessary? > Do we need to actually type-check those? Lambdas are typically > not used in cross-unit interfaces where the type checking is > most needed. > """ > I say trash 'm if there really a problem grammatically. We can still > do the same with nested defs, which are generally more efficient > given the choice of doing either anyway. Vote tallied. > re: > """ > One other possible syntax change would be to introduce an interface > keyword. This would lexically replace the class keyword in Python > code, and its body > would be similar to a class body except that attributes could be > declared but not assigned and functions could have doc strings, but no > code body. I'm not sure that > this buys us anything over just using a regular class and adhering to > those rules. > """ > > I like the interface keyword, even if it only happens inside a decl > statement so it won't have to be a keyword: the reason is that I can > see it being much simpler to name interfaces and classes the same > thing in many cases where ``KlassInterface'' would just jumble up the > namespace with two different names. Basically, I'd like the ability > to follow C-style declarations here where you can name interfaces the > same thing as the actual thing, and I think that should extend to > classes. It would help to tighten the bond between the frequent 1-1 > relationship between interfaces and classes. C/C++ has distinct namespaces for certain names. That option isn't really available to Python. When a name is used, it resolves to a particular object. Within an ordered set of namespaces (local, global, builtins), name resolution is context independent. A name in an expression is the same as one in a function argument. I don't think that we can provide for an interface and a class to share a name. Cheers, -g -- Greg Stein, http://www.lyra.org/ From gstein@lyra.org Wed Jan 5 04:37:50 2000 From: gstein@lyra.org (Greg Stein) Date: Tue, 4 Jan 2000 20:37:50 -0800 (PST) Subject: [Types-sig] updated grammar (was: updated proposal) In-Reply-To: Message-ID: On Mon, 3 Jan 2000, Tony Lownds wrote: > I just tried to compile Python 1.5.2 with your syntax changes. I got > pretty far but your syntax needed a bit of tweaking. Here is a tweaked > version of the "type declarators" code snippet that worked[1]: Excellent! Very cool. > > # first NAME should be: 'member', 'var', 'class', etc > decl_stmt: 'decl' NAME NAME ':' typedecl > typedecl: item_tdecl ('or' item_tdecl)* > item_tdecl: param_tdecl | list_tdecl | dict_tdecl | func_tdecl | > tuple_tdecl > param_tdecl: dotted_name ['(' arglist_tdecl ')'] > tuple_tdecl: '(' typedecl (',' typedecl)* [','] ')' > list_tdecl: '[' typedecl ']' > dict_tdecl: '{' typedecl ':' typedecl '}' > func_tdecl: 'def' '(' [varargs_tdecl] ')' '->' typedecl > varargs_tdecl: (arg_tdecl ',')* ('*' ':' typedecl [',' '**' ':' typedecl] > | '**' ':' typedecl) | arg_tdecl (',' arg_tdecl)* [','] > arg_tdecl: typedecl > > > The changes are: > - the grouping alternative in item_tdecl conflicted with tuple_tdecl, so > tuple_tdecl handles grouping and tuples Ah. Right. I added the grouping when I wrote the interface parser in my prototype and realized that: decl member foo: def()->Int or String required parentheses so that a person could differentiate between the two possible meanings. In the prototype parser, there is no tuple syntax :-). I copied over the grouping to the type-proposal without realizing the conflict. Well... I will note that Guido resolves grouping vs. tuple construction at compile time, too :-) > - arg_tdecl cannot have a name, just a type. I don't see a way to have: > > arg_tdecl: [NAME ':'] typedecl > > in python's parser, because typedecl can start with a > NAME via dotted_name) so the rule makes an ambiguous DFA. You > can have: > > arg_tdecl: typedecl [':' typedecl] > > and constrain the first typedecl in the AST-consuming code. I think I can re-jigger the grammar. I'll work on that later tonite. > I'm going to continue to add the syntax you've laid out. I've > independently been writing grammar for in-line function definitions stuff > so that'll be the next piece. Woo! That is awesome. Adding decl statements and typedecls should have not required a compile.c change (unless you use them in source code), as you probably noted. Similarly, adding the function definition changes and "typedef" operator would not require compile.c changes unless you use those things in source code. Adding the type-assert operator will require a change whether you use the operator or not (because it introduces "or_test" into the parse tree). > However, I'd like to hear from people who have an opinion if using a > modified build is even adviseable. My recent experience suggests it is not > because when you change the grammer you often trip up the compiler and end > up with a broken python. Only with the type-assert operator. The others are quite safe, unless you truly feed that source into the Python compiler (rather than just the parser). I would *really* like to see a patch set for your changes. That would be awesome. I'd also like to make them available via my web pages. > If it *is* adviseable, then the thing to do is to completely specify the > syntax and then make a Grammar and compile.c that works with current > syntax (ignoring any type declarations). I think it is. It is very helpful to actually work with code, rather than sit around and talk for weeks on end. Gotta get in an really try things! > If it is *not* adviseable, then the thing to do is to make check.py go > through an interface instead of accessing the parser, token and symbol > modules directly, and look for another way to turn Python programs into > parse trees. I don't want to replicate the Python parser :-) ... instead, we would have to rely entirely on external files and/or string-based or functional replacements for the syntax (as described in my recent feedback to the PyDL RFC). > Either way I'm pretty interested in getting check.py to work on a modified > Python syntax, if only to test ideas. Me too! I will note that check.py could easily handle working with the standard parser or you modified parser. It can degrade gracefully. Cheers, -g -- Greg Stein, http://www.lyra.org/ From m.faassen@vet.uu.nl Wed Jan 5 10:02:33 2000 From: m.faassen@vet.uu.nl (Martijn Faassen) Date: Wed, 05 Jan 2000 11:02:33 +0100 Subject: [Types-sig] check.py (was: PyDL RFC 0.02) References: <000601bf528c$70a62920$a02d153f@tim> <3871F51B.6CAE9870@vet.uu.nl> <14450.26410.693253.653903@beluga.mojam.com> Message-ID: <387316B9.15B0C740@vet.uu.nl> Skip Montanaro wrote: > > Martijn> def foo(a the Integer, b the String, c the Foo, d the Any) the Integer: > Martijn> e = d the Integer > Martijn> return e > > Martijn> I could definitely live with that one. Doesn't have the > Martijn> semantic confusion that 'as' have and it seems readable. > > I have to toss out the barb that when the most frequently used token in my > Python code is "the" or "as" it will be time to find another language... :) > (Someone please reassure me by stating emphatically that all this extra > stuff will never be required!) Type-checking is optional, so all the 'as', '!', ':', 'the' or whatever it'll be will never be required. I just thought 'the' looked better than the ':' approach (but I may change my mind tomorrow): def foo(a: Integer, b: String, c: Foo, d: Any)->Integer: e = d ! Integer return e Yeah, I may change my mind tomorrow. I just thought 'the' was cute. :) Regards, Martijn From m.faassen@vet.uu.nl Wed Jan 5 10:31:02 2000 From: m.faassen@vet.uu.nl (Martijn Faassen) Date: Wed, 05 Jan 2000 11:31:02 +0100 Subject: [Types-sig] Re: feedback, round 2 (LONG) (Re: PyDL RFC 0.4) References: <38725B49.831FE79C@prescod.net> Message-ID: <38731D66.C8BB7452@vet.uu.nl> Paul Prescod wrote: > Greg Stein wrote: [snip snip] > Okay, but C revolves around a concept of types. Python (arguably) is > supposed to revolve around a concept of interfaces. "I don't care if you > hand me a file thing, as long as it BEHAVES like a file thing." Our > static type checker is really a static interface checker. For the rare > case when we really care about types, we'll use the implied, > implementation-specific interface. I agree with this very strongly; I reached about the same conclusion before christmas, but didn't find time to clean up my text and post it. It is interesting that you people seem to be going in the same direction -- this would imply we're on the right track. [snip] > > We should. I want to parameterize classes. Don't force me to extract an > > interface from my class definition -- I want an implicit, parameterized > > interface derived from my class definition and its inline declarations. > > I see this as: > > "Future Directions: At some point in the future, PyDL files will likely > be generated from source code using a combination of declarations within > Python code and some sorts of interface deduction and inferencing based > on various kinds of assignment." > > Note that I have also provided no syntax for specifying the types of > functions in the function declarations. You make a separate declaration > with the types. My take is that all classes should imply an interface. I'm going back to my pre-christmas thoughts here. :) If we split the types namespace from the variable namespace (I'm in favor of this), then the implied interface of the class can be the same name (in the same module context etc) as the name of the class. So: class Foo: ... implies implicit interface Foo which is extracted from all the inline declarations of Foo. > > The basic premise is that I will use classes as *both* an interface > > definition and an implementation. To that end, I want all the features of > > a standard inteface definition to be available through my class > > definition. > > But an interface is an interface. A class is a type. A type can be used > as an "implementation specific interface" but this is a bad habit that > leads to weirdness like functions that work on file objects but blow up > on FTP download streams. Again I'm going back to my previous thoughts: you can actually derive interface definitions almost 'for free'. Now we do explicitly name interfaces with something like 'conforms': class A conforms GetPutInterface: decl member a: Int decl member b: Int def get()->Int: ... def put(d: Int): ... def bar(): ... class B conforms GetPutInterface: decl member a: Int decl member c: Int def get()->Int: ... def put(d: Int): ... def foo(): ... Now the structure of GetPutInterface can be automatically derived from the union of the interfaces of these two classes. You *don't* explicitly define GetPutInterface anywhere (though you can): interface GetPutInterface: decl member a: Int def get()->Int def put(d: Int) So, you get the common interface with minimal changes to the code. Of course in many cases you do want to say what the interface is explicitly; otherwise you could (by mistake) limit union of the interfaces too much (even to nothing at all). You make sure a constructed interface is not too large by making sure you have a class with a minimal interface that conforms to the same interface. You can make sure a constructed interface isn't too small by explicitly stating the interface. But you don't have to explicitly state the interface. This makes current Python code very easy to adapt to interfaces, you can add interfaces in an incremental way and slowly 'harden' your code to use explicit interfaces. But you don't have to; you just have to say a class conforms to an interface, and they're there. So far my last century thoughts. :) > I agree that there should someday be a syntax for extracting an > implementation *independent* interface from a class as a convenience but > then we need to get into issues of "public/private" and so forth. After > all, it will almost never be the case that a class's interface will be > 100% identical to its implementation signature. My scheme also neatly avoids this problem. I initially thought you'd need access control as well, but if you construct interfaces as the union of two class interfaces (the ones always implied by the class that have the same name as the class), your interface is exactly as big as it has to be (unless of course you did something wrong). It doesn't include 'private' data that's only used in the implementation, at least not as easily. It may of course be possible that data or functions of the exact same name and signature are used very differently in two classes. In that case you do want the ability to explicitly define an interface. Anyway, summary: * each class implies an interface with the same name * each class can be explicitly pecified to conform to an interface of some name * this interface does not have to be explicitly defined somewhere; it can be derived from the union of all classes that state they conform to this interface * this interface *can* be explicitly defined somewhere. * in this scheme, we're dealing with 'static interface checking'. Regards, Martijn From m.faassen@vet.uu.nl Wed Jan 5 10:32:55 2000 From: m.faassen@vet.uu.nl (Martijn Faassen) Date: Wed, 05 Jan 2000 11:32:55 +0100 Subject: [Types-sig] OPINIONS: Warnings References: <38723F11.B5939730@prescod.net> Message-ID: <38731DD7.9E81951F@vet.uu.nl> Paul Prescod wrote: [snip] > Therefore we should banish warnings from our lexicon for the purposes of > this discussion. > > Opinions? I'm inclined to agree for much the same reasons as the ones you listed. me-too-ly yours, Martijn From skaller@maxtal.com.au Wed Jan 5 13:38:22 2000 From: skaller@maxtal.com.au (skaller) Date: Thu, 06 Jan 2000 00:38:22 +1100 Subject: [Types-sig] updated proposal (fwd) References: Message-ID: <3873494E.F78699C@maxtal.com.au> > Given the following code: > > decl a: Int > b = a ! String > > The compiler will know that it will always fail. So it can flag it. In the > following: > > decl a: Any > b = a ! String > > It has no idea. So it lets it pass, to be evaluated at runtime. The > operator *is* a runtime operator. The reason it is important to the > compile-time checker, however, is that the checker now knows that "b" is a > String. It cannot be anything else (or the exception would prevent the > assignment). More precisely: b is a string from the point at which b is bound to the checked expression, at least until: 1) 'end of block' 2) An exec statement is seen 3) An assignment to b 4) If b is global, then a function call 5) the module dictionary is fiddled with 1) needs to be made more precise: at places like the end of a conditional component in which the assignment may be enclosed, it is possible that a control path originating _before_ the checked assignment branching to a point afterwards, may leave 'b' bound to some non-string object, or even unbound. The above conditions should be 'reasonbly' easy to check; except (5), must be taken as a precondition (short of more sophisticated control flow analysis). Note these conditions won't give the best results: there must be a way of determining when to 'forget' type information, as well as deduce it. This 'way' must be conservative: if there's any doubt, it is always safe to 'forget' type information. Note that a function call can invalidate type assumptions in two ways: by declaring b global and assigning to it, or, by indirectly refering to b by a module.attr = value assignment (or, worse, a dictionary fiddle). IMHO: Because function calls are used a lot, global analysis by _inlining_ functions will improve type safety enormously: point (4) is a real killer: note it _only_ applies to global symbols (not function local ones). {** probably also class instance attribute typing] The function call problem would be alleviated by banning module.attr = value and 'global' statements, but the latter are reasonably easy to detect. The former are really hard to detect (at compile time). I'd love to see a better algorithm for 'forgetting' type information. -- John (Max) Skaller, mailto:skaller@maxtal.com.au 10/1 Toxteth Rd Glebe NSW 2037 Australia homepage: http://www.maxtal.com.au/~skaller voice: 61-2-9660-0850 From paul@prescod.net Wed Jan 5 19:35:29 2000 From: paul@prescod.net (Paul Prescod) Date: Wed, 05 Jan 2000 14:35:29 -0500 Subject: [Types-sig] Re: syntax and compile/run -time References: <3872B419.13164EA4@prescod.net> <20000104224240.A98858@chronis.pobox.com> Message-ID: <387372D1.DBCA9A6@prescod.net> scott wrote: > > is ok at compile time, but may or may not cause an error at runtime. > > I'm not against keeping the same basic syntax, but I do believe it > should be made clear *in the decl statement* that it is not intended > to be part of the compile-time stuff. If the programmer indeed > intends that (and they really might), they need to be told about it in > a way that is more obvious than a warning. I agree with you 100% philosophically and if a new keyword "runtime" is an acceptable compromise then I would be willing to do it. It's an added complication but only people trying to do complicated stuff would bump into it. I think that the right place is on the particular argument that cannot be evaluated at compile time: a = GoFetchMeAnInterface() decl foo as def( arg: runtime a ) -> String decl foo as def( arg: runtime GoFetchMeAnInterface() ) -> String We might think of extending this keyword to other contexts where things that would otherwise be expected to be compile time: runtime import somedbadapter The compiler wouldn't try to look up somedbadapter at compile time and names from that namespace would be invisible at compile time. Greg, is this a workable compromise? I would like to say that one could think of typedef and runtime as inverses but it doesn't quite work out...Greg's model is that "typedef" gives access to new syntax, but is sometimes evaluated at runtime. Paul Prescod From scott@chronis.pobox.com Wed Jan 5 19:48:39 2000 From: scott@chronis.pobox.com (scott) Date: Wed, 5 Jan 2000 14:48:39 -0500 Subject: [Types-sig] updated proposal (fwd) In-Reply-To: <3873494E.F78699C@maxtal.com.au> References: <3873494E.F78699C@maxtal.com.au> Message-ID: <20000105144839.B4237@chronis.pobox.com> On Thu, Jan 06, 2000 at 12:38:22AM +1100, skaller wrote: > > > Given the following code: > > > > decl a: Int > > b = a ! String > > > > The compiler will know that it will always fail. So it can flag it. In the > > following: > > > > decl a: Any > > b = a ! String > > > > It has no idea. So it lets it pass, to be evaluated at runtime. The > > operator *is* a runtime operator. The reason it is important to the > > compile-time checker, however, is that the checker now knows that "b" is a > > String. It cannot be anything else (or the exception would prevent the > > assignment). OK, what I see as wishy-washy about the behavior of the thing is that it does one thing at compile time and another at run time. As skaller writes below, whatever it does at compile is really complex, in fact, IMO too complex to really make any use out of at compile time at all. To me, if the static typing information states that a is 'Any', then if at any other point, a is considered a string, then the static typing error is flagged because it is not type-safe in terms of the presented static typing information. In other words, the static type information should be a closed system. It's *much* easier to work with that way -- both in terms of implementing it and using it. I don't think this should be deferred till run time by the static type checker. Instead, it seems to me that: decl a: Any decl def f(a: String) -> Whatever a ! String f(a) is not type safe... especially given the complexity of understanding the implications of `!' at compile-time. If it is indeed essentially a run time operator, then it should not affect the compile-time checking. Then, you probably want to know, what do we do with union types? a union type should be a distinct type, it's not really an expression of 'one of a or b or ... or n' in the static compile time system as it is a statement that 'any of a or b or ... or n'. must be true for everything we can see at compile time. So, when I stated before that it would be nice if we could take the type-assert operator and make it flag all errors at compile time, even if it meant following a few simple restrictions to gain that feature, I'd like to re-phrase that as the type-assert operator needs to either work completely at compile time, or completely at run time. It seems dubious that we can work out a simple enough system to use the thing at compile time, though it might be possible. As a run-time only tool, it doesn't seem to offer substantially more than asserts do now. Furthermore, the tools provided to work with typing information at run-time should in no way alter the behavior of the compile-time system. At run time, we should be able to read the results of any compile time checking, and even create/overwrite new things like typedecl objects, but that doing so simply should not affect how the type system behaves at compile time. scott > > More precisely: b is a string from the point at which b is > bound to the checked expression, at least until: > > 1) 'end of block' > 2) An exec statement is seen > 3) An assignment to b > 4) If b is global, then a function call > 5) the module dictionary is fiddled with > > 1) needs to be made more precise: at places like > the end of a conditional component in which > the assignment may be enclosed, it is possible > that a control path originating _before_ the > checked assignment branching to a point > afterwards, may leave 'b' bound to some non-string > object, or even unbound. > > The above conditions should be 'reasonbly' easy to check; > except (5), must be taken as a precondition (short of > more sophisticated control flow analysis). > > Note these conditions won't give the best results: there must be > a way of determining when to 'forget' type information, > as well as deduce it. This 'way' must be conservative: > if there's any doubt, it is always safe to 'forget' type information. > > Note that a function call can invalidate type assumptions > in two ways: by declaring b global and assigning to it, > or, by indirectly refering to b by a module.attr = value > assignment (or, worse, a dictionary fiddle). > > IMHO: Because function calls are used a lot, global analysis > by _inlining_ functions will improve type safety enormously: > point (4) is a real killer: note it _only_ applies to > global symbols (not function local ones). > {** probably also class instance attribute typing] > > The function call problem would be alleviated by > banning module.attr = value and 'global' statements, > but the latter are reasonably easy to detect. > The former are really hard to detect (at compile time). > > I'd love to see a better algorithm for 'forgetting' > type information. > > -- > John (Max) Skaller, mailto:skaller@maxtal.com.au > 10/1 Toxteth Rd Glebe NSW 2037 Australia > homepage: http://www.maxtal.com.au/~skaller > voice: 61-2-9660-0850 > > _______________________________________________ > Types-SIG mailing list > Types-SIG@python.org > http://www.python.org/mailman/listinfo/types-sig From scott@chronis.pobox.com Wed Jan 5 20:15:34 2000 From: scott@chronis.pobox.com (scott) Date: Wed, 5 Jan 2000 15:15:34 -0500 Subject: [scott@chronis.pobox.com: Re: [Types-sig] updated proposal (fwd)] Message-ID: <20000105151534.A4782@chronis.pobox.com> In my usual habbit, I forgot to address this to the list. scott ----- Forwarded message from scott ----- Date: Wed, 5 Jan 2000 15:11:02 -0500 From: scott To: Greg Stein Subject: Re: [Types-sig] updated proposal (fwd) X-Mailer: Mutt 0.95.7i In-Reply-To: On Tue, Jan 04, 2000 at 08:26:35PM -0800, Greg Stein wrote: > Great comments. Thanx for the feedback. thanks for the code and web page :) > > what operator to use as type-assert operator: > > > > I prefer 'isa' or 'asa'. If I still used emacs, I'd _hate_ to have to > > wait for python-mode.el to catch up to understanding the colons in > > type-assert operators, short of that, I prefer the colon, it looks > > familiar. > > I don't think that anybody has suggested using the colon for the > *operator*. It is grammatically ambiguous. > > Could you clarify your position, and I'll mark the tally in the proposal. > Thx. I was confused about what exactly was meant by type-assert operator. Please disregard :) [...] > > regarding a new namespace to place interface/type info into... I'd > > suggest leaving that open for the time being, or just defining that > > access is available at runtime via a function call such as: > > > > interfaces() -> [ all known interfaces in current namespace ] > > interfaces(x: Any) -> [ all known interfaces that x implements ] > > > > this would allow 'implementors' to put the darn info wherever they > > want and define a flexible enough interface to getting the interfaces > > to allow for variable underlying implementations. (sorry for > > mouthful...) > > Interesting thought, but I feel confident that we don't need another > namespace, so I don't really see a reason to make the above compromise. how would it compromise anything about where type info is stored? [...] > > There should be interfaces for all objects, and those we can't make > > right off the bat we should try to make sure we don't put off in such > > a way that they will become unduly tricky down the road. I think this > > is a key concept we should strive for... in the long run, all built-in > > objects should have a set of interfaces, and those interfaces should > > be defined in terms of atomic (resolved) types wherever possible. > > Agreed. > > I've been thinking about this quite a bit, so that I can add more code to > the type checker to properly deal with operators (e.g. addition). Let's > say you have the following interface: > > interface Foo: > decl member __add__: def(self:Foo, value:Foo)->Foo > > Class instances supporting the Foo interface are capable of being added. > They *also* have an "__add__" attribute. > > Consider a builtin type: it can be added, but it does not have an > "__add__" attribute. How do we resolve this? I believe we do it this way: > > interface Int: > decl intrinsic __add__: def(self:Int, value:Int)->Int > > This tells the checker that Int understands addition, but it does NOT > actually have an __add__ attribute. > > Thoughts? I like it. nice easy way to add lots of interface info to built-in objects. One question, though Is there built-in functionality that would be useful to consider as part of an interface that does not have a corresponding '__magic__' object in python? I can't hink of any off-hand, but it would seem prudent to pose the question. > > re: > > """ > > One other possible syntax change would be to introduce an interface > > keyword. This would lexically replace the class keyword in Python > > code, and its body > > would be similar to a class body except that attributes could be > > declared but not assigned and functions could have doc strings, but no > > code body. I'm not sure that > > this buys us anything over just using a regular class and adhering to > > those rules. > > """ > > > > I like the interface keyword, even if it only happens inside a decl > > statement so it won't have to be a keyword: the reason is that I can > > see it being much simpler to name interfaces and classes the same > > thing in many cases where ``KlassInterface'' would just jumble up the > > namespace with two different names. Basically, I'd like the ability > > to follow C-style declarations here where you can name interfaces the > > same thing as the actual thing, and I think that should extend to > > classes. It would help to tighten the bond between the frequent 1-1 > > relationship between interfaces and classes. > > C/C++ has distinct namespaces for certain names. That option isn't really > available to Python. When a name is used, it resolves to a particular > object. Within an ordered set of namespaces (local, global, builtins), > name resolution is context independent. A name in an expression is the > same as one in a function argument. > > I don't think that we can provide for an interface and a class to share a > name. hmm, i see the complication, and could live with this either way, but I'm not convinced that there does not exist the possibilty of a way to make interfaces and classes share names that is reasonable, just that it's unlikely that one exists. thanks for clarifying. scott ----- End forwarded message ----- From mengx@nielsenmedia.com Wed Jan 5 20:33:37 2000 From: mengx@nielsenmedia.com (mengx@nielsenmedia.com) Date: Wed, 5 Jan 2000 15:33:37 -0500 (EST) Subject: [Types-sig] Re: syntax and compile/run -time Message-ID: <200001052033.PAA19771@p5mts.nielsenmedia.com> Hi, First of all, I hope the flooding of the mailing list was not caused by my cynical remarks during the last millienum :-) > decl foo as def( arg: runtime a ) -> String > > decl foo as def( arg: runtime GoFetchMeAnInterface() ) -> String > > We might think of extending this keyword to other contexts where things > that would otherwise be expected to be compile time: > If using docstring or a special varible name like "types = {a:"int"}" are of some trouble somehow, could it be possible to use the following format ? class aClass: pass def foo_type_string(a,b_type_int, c_type_file,d_type_aClass): return c.read() a="a" b=1 c=open('file') d = aClass() foo(a,b,c,d) in a given scope, b_type_int declares b as "int", after this, interpreter/compiler knows that b is of interger type. "a" is determined at runtime The "_type_sometype" seems visually uncomfortable at first but I feel it better than "$::->" and any other non python-compliant English; It is also optional. Of course I know it is not easy to implement it as well as other suggestions :-) I did not follow closely with the discussion so please ignore my comment if it is irrelevant. Thanks -Ted Meng From scott@chronis.pobox.com Wed Jan 5 20:42:54 2000 From: scott@chronis.pobox.com (scott) Date: Wed, 5 Jan 2000 15:42:54 -0500 Subject: [Types-sig] updated proposal (fwd) In-Reply-To: <20000105144839.B4237@chronis.pobox.com> References: <3873494E.F78699C@maxtal.com.au> <20000105144839.B4237@chronis.pobox.com> Message-ID: <20000105154254.B4782@chronis.pobox.com> On Wed, Jan 05, 2000 at 02:48:39PM -0500, scott wrote: > On Thu, Jan 06, 2000 at 12:38:22AM +1100, skaller wrote: > > > > > Given the following code: > > > > > > decl a: Int > > > b = a ! String > > > > > > The compiler will know that it will always fail. So it can flag it. In the > > > following: > > > > > > decl a: Any > > > b = a ! String > > > > > > It has no idea. So it lets it pass, to be evaluated at runtime. The > > > operator *is* a runtime operator. The reason it is important to the > > > compile-time checker, however, is that the checker now knows that "b" is a > > > String. It cannot be anything else (or the exception would prevent the > > > assignment). > > OK, what I see as wishy-washy about the behavior of the thing is that > it does one thing at compile time and another at run time. As skaller > writes below, whatever it does at compile is really complex, in fact, > IMO too complex to really make any use out of at compile time at all. > [...] > Then, you probably want to know, what do we do with union types? a > union type should be a distinct type, it's not really an expression of > > 'one of a or b or ... or n' > > in the static compile time system as it is a statement that > > 'any of a or b or ... or n'. > > must be true for everything we can see at compile time. just to clarify: I see any type declaration statement as distinct from not declaring anything. If I declare somevar as having type 'Any' it means that it is the Any type (which in itself doesn't enforce anything other than the variable cannot be used in a way that asserts a particular type). When something is not declared, it can be deduced from a relatively predictable algorithm, but that deduced type is one which allows the more permissive 'one of a or b or ... or n'. Declared types seem more useful as 'always any of a or b or ... or n' to me. If we wanted the 'one of...' behavior in type declarations, I guess I'd be more inclined to call that 'xor' than 'or'. Given a choice between the two, I'd prefer the 'always any of...' semantics. scott From gstein@lyra.org Wed Jan 5 21:46:43 2000 From: gstein@lyra.org (Greg Stein) Date: Wed, 5 Jan 2000 13:46:43 -0800 (PST) Subject: [Types-sig] updated proposal (fwd) Message-ID: Forwarding again. Maybe scott will learn the "Reply to All" button :-) ---------- Forwarded message ---------- Date: Wed, 5 Jan 2000 15:11:02 -0500 From: scott To: Greg Stein Subject: Re: [Types-sig] updated proposal (fwd) On Tue, Jan 04, 2000 at 08:26:35PM -0800, Greg Stein wrote: > Great comments. Thanx for the feedback. thanks for the code and web page :) > > what operator to use as type-assert operator: > > > > I prefer 'isa' or 'asa'. If I still used emacs, I'd _hate_ to have to > > wait for python-mode.el to catch up to understanding the colons in > > type-assert operators, short of that, I prefer the colon, it looks > > familiar. > > I don't think that anybody has suggested using the colon for the > *operator*. It is grammatically ambiguous. > > Could you clarify your position, and I'll mark the tally in the proposal. > Thx. I was confused about what exactly was meant by type-assert operator. Please disregard :) [...] > > regarding a new namespace to place interface/type info into... I'd > > suggest leaving that open for the time being, or just defining that > > access is available at runtime via a function call such as: > > > > interfaces() -> [ all known interfaces in current namespace ] > > interfaces(x: Any) -> [ all known interfaces that x implements ] > > > > this would allow 'implementors' to put the darn info wherever they > > want and define a flexible enough interface to getting the interfaces > > to allow for variable underlying implementations. (sorry for > > mouthful...) > > Interesting thought, but I feel confident that we don't need another > namespace, so I don't really see a reason to make the above compromise. how would it compromise anything about where type info is stored? [...] > > There should be interfaces for all objects, and those we can't make > > right off the bat we should try to make sure we don't put off in such > > a way that they will become unduly tricky down the road. I think this > > is a key concept we should strive for... in the long run, all built-in > > objects should have a set of interfaces, and those interfaces should > > be defined in terms of atomic (resolved) types wherever possible. > > Agreed. > > I've been thinking about this quite a bit, so that I can add more code to > the type checker to properly deal with operators (e.g. addition). Let's > say you have the following interface: > > interface Foo: > decl member __add__: def(self:Foo, value:Foo)->Foo > > Class instances supporting the Foo interface are capable of being added. > They *also* have an "__add__" attribute. > > Consider a builtin type: it can be added, but it does not have an > "__add__" attribute. How do we resolve this? I believe we do it this way: > > interface Int: > decl intrinsic __add__: def(self:Int, value:Int)->Int > > This tells the checker that Int understands addition, but it does NOT > actually have an __add__ attribute. > > Thoughts? I like it. nice easy way to add lots of interface info to built-in objects. One question, though Is there built-in functionality that would be useful to consider as part of an interface that does not have a corresponding '__magic__' object in python? I can't hink of any off-hand, but it would seem prudent to pose the question. > > re: > > """ > > One other possible syntax change would be to introduce an interface > > keyword. This would lexically replace the class keyword in Python > > code, and its body > > would be similar to a class body except that attributes could be > > declared but not assigned and functions could have doc strings, but no > > code body. I'm not sure that > > this buys us anything over just using a regular class and adhering to > > those rules. > > """ > > > > I like the interface keyword, even if it only happens inside a decl > > statement so it won't have to be a keyword: the reason is that I can > > see it being much simpler to name interfaces and classes the same > > thing in many cases where ``KlassInterface'' would just jumble up the > > namespace with two different names. Basically, I'd like the ability > > to follow C-style declarations here where you can name interfaces the > > same thing as the actual thing, and I think that should extend to > > classes. It would help to tighten the bond between the frequent 1-1 > > relationship between interfaces and classes. > > C/C++ has distinct namespaces for certain names. That option isn't really > available to Python. When a name is used, it resolves to a particular > object. Within an ordered set of namespaces (local, global, builtins), > name resolution is context independent. A name in an expression is the > same as one in a function argument. > > I don't think that we can provide for an interface and a class to share a > name. hmm, i see the complication, and could live with this either way, but I'm not convinced that there does not exist the possibilty of a way to make interfaces and classes share names that is reasonable, just that it's unlikely that one exists. thanks for clarifying. scott From gstein@lyra.org Wed Jan 5 21:53:41 2000 From: gstein@lyra.org (Greg Stein) Date: Wed, 5 Jan 2000 13:53:41 -0800 (PST) Subject: [Types-sig] updated proposal (fwd)] In-Reply-To: <20000105151534.A4782@chronis.pobox.com> Message-ID: On Wed, 5 Jan 2000, scott wrote: > In my usual habbit, I forgot to address this to the list. Dang. I just forwarded the original before seeing this... ah well. >... > > > regarding a new namespace to place interface/type info into... I'd > > > suggest leaving that open for the time being, or just defining that > > > access is available at runtime via a function call such as: > > > > > > interfaces() -> [ all known interfaces in current namespace ] > > > interfaces(x: Any) -> [ all known interfaces that x implements ] > > > > > > this would allow 'implementors' to put the darn info wherever they > > > want and define a flexible enough interface to getting the interfaces > > > to allow for variable underlying implementations. (sorry for > > > mouthful...) > > > > Interesting thought, but I feel confident that we don't need another > > namespace, so I don't really see a reason to make the above compromise. > > how would it compromise anything about where type info is stored? Sorry. I didn't mean that the info would be compromised... I meant that I would not compromise my position :-) In other words, I'm feeling righteous enough :-) that I think we don't need a new namespace or an interfaces() function for this problem. >... > > Consider a builtin type: it can be added, but it does not have an > > "__add__" attribute. How do we resolve this? I believe we do it this way: > > > > interface Int: > > decl intrinsic __add__: def(self:Int, value:Int)->Int > > > > This tells the checker that Int understands addition, but it does NOT > > actually have an __add__ attribute. > > > > Thoughts? > > I like it. nice easy way to add lots of interface info to built-in > objects. One question, though Is there built-in functionality that > would be useful to consider as part of an interface that does not have > a corresponding '__magic__' object in python? I can't hink of any > off-hand, but it would seem prudent to pose the question. I with you -- I don't believe so. A thorough review will turn stuff up, though. If there *are* any, then we could just do something like: decl intrinsic bind_method: def(...)->BoundMethod This would probably work out quite well. Cheers, -g -- Greg Stein, http://www.lyra.org/ From gstein@lyra.org Wed Jan 5 22:42:08 2000 From: gstein@lyra.org (Greg Stein) Date: Wed, 5 Jan 2000 14:42:08 -0800 (PST) Subject: [Types-sig] compile/run -time (was: syntax and compile/run -time) In-Reply-To: <387372D1.DBCA9A6@prescod.net> Message-ID: On Wed, 5 Jan 2000, Paul Prescod wrote: >... > I agree with you 100% philosophically and if a new keyword "runtime" is > an acceptable compromise then I would be willing to do it. It's an added > complication but only people trying to do complicated stuff would bump > into it. The "runtime" keyword presumes too much knowledge on the programmer's part. How are they supposed to know the full bounds and limitations of the type checker? If you say, "you must insert 'runtime' in places where the type checker can't figure out something at compile-time," then they're next question will be, "okay. when is that?" Instead, allow all the forms and make the type-checker do its best. The code will always work properly. If somebody marks a function as "typesafe", then the checker should raise an error if it cannot determine a particular typedecl condition (like with the "a=get_an_interface()" example). If a function is not marked as "typesafe", then we will not have lost any functionality over 1.5, but we certainly have gained: the argument will be checked on function entry. a = get_an_interface() def foo(x: a): ... That is much better than what I can do today. The type-checker can't deal with it, so it leaves it to the run time. *IF* the user turns on -Wall, or maybe just -Wunknown-typedecl, then the checker will warn/error on this kind of construct. If the user does: a = get_an_interface() typesafe def foo(x: a): ... The type-checker will flat-out refuse to allow this. Allowing the complex typing construct cannot hurt us -- it can help us. Leave the flexibility to the programmer to decide how they want to handle these things -- stop trying to legislate so hard. >... > We might think of extending this keyword to other contexts where things > that would otherwise be expected to be compile time: > > runtime import somedbadapter > > The compiler wouldn't try to look up somedbadapter at compile time and > names from that namespace would be invisible at compile time. Presumably "somedbadapter" has no interface information, so why add the "runtime" to it? The two situations are the same -- whether it doesn't find it or you tell it not to look for it. Granted, if the user happens to use -Wunknown-modules, then they get a warn/error on an import where no interface information can be produced. > Greg, is this a workable compromise? It requires the programmer to know too much about the type-checker. I think we have the runtime/compile-time issue covered without this. > I would like to say that one could think of typedef and runtime as > inverses but it doesn't quite work out...Greg's model is that "typedef" > gives access to new syntax, but is sometimes evaluated at runtime. a = typedef Int or String is always evaluated at runtime. It creates a typedecl object and assigns that to "a". The question is whether the type-checker can do anything with it at compile-time. In most cases: yes, it can understand the situation. In atypical cases: no, it can't and bugger on those people. Cheers, -g -- Greg Stein, http://www.lyra.org/ From gstein@lyra.org Wed Jan 5 23:07:19 2000 From: gstein@lyra.org (Greg Stein) Date: Wed, 5 Jan 2000 15:07:19 -0800 (PST) Subject: [Types-sig] type-asserts and check/run -time (was: updated proposal) In-Reply-To: <20000105144839.B4237@chronis.pobox.com> Message-ID: On Wed, 5 Jan 2000, scott wrote: > On Thu, Jan 06, 2000 at 12:38:22AM +1100, skaller wrote: > > > Given the following code: > > > > > > decl a: Int > > > b = a ! String > > > > > > The compiler will know that it will always fail. So it can flag it. In the > > > following: > > > > > > decl a: Any > > > b = a ! String > > > > > > It has no idea. So it lets it pass, to be evaluated at runtime. The > > > operator *is* a runtime operator. The reason it is important to the > > > compile-time checker, however, is that the checker now knows that "b" is a > > > String. It cannot be anything else (or the exception would prevent the > > > assignment). > > OK, what I see as wishy-washy about the behavior of the thing is that > it does one thing at compile time and another at run time. As skaller *everything* is evaluated differently at runtime and compile-time. Everything. a = f(b) A function call doesn't occur at type-check time. Instead, it watches type signatures flow around. At runtime, a function call is performed but there is no type analysis occurring (the types are held by the value objects). I see no problem in defining a runtime behavior which has a feature that the type-checker can make use of at check-time. > writes below, whatever it does at compile is really complex, in fact, > IMO too complex to really make any use out of at compile time at all. Yes, it is complex. So? There is nothing unique about this operator. Consider: a = 5 if f(): a = "5" b = a In the above construct, we have to consider the same five conditions that skaller noted. There is nothing special/unique about the type-assert operator. My point here is that the checker must be considering those five *all* the time, for *all* operators, statements, and control structures. This is where Tim Peters will pipe in and say "but if everything is required to be declared, then the problem is reduce and those five conditions to not necessarily (all) apply." We can require a declaration for every single variable that you ever want to use, or we can add some smarts to the checker. I prefer the latter, and put forward my prototype as an example that much can be done already (with no declarations). > To me, if the static typing information states that a is 'Any', then > if at any other point, a is considered a string, then the static > typing error is flagged because it is not type-safe in terms of the > presented static typing information. String is a subtype of Any, so there shouldn't be an error. If the checker wants to make a finer-grained assessment of "a", then it should be able to. Given: decl a: Any a = "string" func_taking_string(a) Strictly speaking, yes: that could be called an error. In reality: I would hope that the checker could figure it out and *not* flag an error. There is no reason to, as we know it meets all type-safety concerns. > In other words, the static type > information should be a closed system. It's *much* easier to work > with that way -- both in terms of implementing it and using it. I don't understand "closed system" in this context. > I don't think this should be deferred till run time by the static type > checker. Instead, it seems to me that: > > decl a: Any > decl def f(a: String) -> Whatever > a ! String > f(a) > > is not type safe... especially given the complexity of understanding > the implications of `!' at compile-time. If it is indeed essentially > a run time operator, then it should not affect the compile-time > checking. The type-checker as I've been coding it (so far) would not remap "a" to a String. The checker sees: ! String Given that: the f(a) will raise an error. A smart type-checker *would* remap "a" for some period and allow the above code to be compiled without flagging an error. And why shouldn't it? You and I know what that code says. Why should we state that the type-checker can never be as smart as us? The whole *point* of the type-checker is for it to be smarter than we are. We are trying to use it to find the places where we are being stupid. >... > So, when I stated before that it would be nice if we could take the > type-assert operator and make it flag all errors at compile time, even > if it meant following a few simple restrictions to gain that feature, > I'd like to re-phrase that as > > the type-assert operator needs to either work completely at compile > time, or completely at run time. It seems dubious that we can work > out a simple enough system to use the thing at compile time, though it > might be possible. As a run-time only tool, it doesn't seem to offer > substantially more than asserts do now. The type-assert operator is defined to *always* operate at runtime[*]. The user should depend upon this. My statement was that a type-checker can detect certain situations: decl a: Any a = 5 b = a ! String Given the above code, I *want* the type-checker to raise an error. I was being an idiot. I assigned an integer, then expected a string. And you are correct: as a runtime tool, it *is* nothing more than an assert. Its reason-to-exist is because the type-checker can gather a LOT of information from it. Its definition of operation also lets us leverage what happens with function arguments: def foo(x: a): some_code() can be written as: def foo(x): x ! a some_code() It is simply that the former is easier to write AND it provides a signature for callers to use. > Furthermore, the tools provided to work with typing information at > run-time should in no way alter the behavior of the compile-time > system. I'm not sure if my above explanations satisfy this. I see the two as distinct operations, with no dependencies between them. The type-checker derives information from what it knows about runtime operation. That is a given. I also believe the bytecodes that are generated will always be the same, despite what the type-checker says about things. [ skaller will pipe in here to say that the compiler can be optimized based on the type information -- true, but I'm not concerned with that aspect right now. ] > At run time, we should be able to read the results of any > compile time checking, and even create/overwrite new things like > typedecl objects, but that doing so simply should not affect how the > type system behaves at compile time. Agreed. Cheers, -g For reference, skaller writes: > > More precisely: b is a string from the point at which b is > > bound to the checked expression, at least until: > > > > 1) 'end of block' > > 2) An exec statement is seen > > 3) An assignment to b > > 4) If b is global, then a function call > > 5) the module dictionary is fiddled with -- Greg Stein, http://www.lyra.org/ From gstein@lyra.org Wed Jan 5 23:09:36 2000 From: gstein@lyra.org (Greg Stein) Date: Wed, 5 Jan 2000 15:09:36 -0800 (PST) Subject: [Types-sig] Re: syntax and compile/run -time In-Reply-To: <200001052033.PAA19771@p5mts.nielsenmedia.com> Message-ID: On Wed, 5 Jan 2000 mengx@nielsenmedia.com wrote: >... > > decl foo as def( arg: runtime a ) -> String > > > > decl foo as def( arg: runtime GoFetchMeAnInterface() ) -> String > > > > We might think of extending this keyword to other contexts where things > > that would otherwise be expected to be compile time: > > If using docstring or a special varible name like "types = {a:"int"}" > are of some trouble somehow, could it be possible to use the following format ? We are discussing how to change the Python syntax. We are not restricted to backwards-compatible changes in our design. Cheers, -g -- Greg Stein, http://www.lyra.org/ From gstein@lyra.org Wed Jan 5 23:14:44 2000 From: gstein@lyra.org (Greg Stein) Date: Wed, 5 Jan 2000 15:14:44 -0800 (PST) Subject: [Types-sig] type-asserts and check/run -time In-Reply-To: Message-ID: Dang. Forgot to fill in my footnote... On Wed, 5 Jan 2000, Greg Stein wrote: >... > The type-assert operator is defined to *always* operate at runtime[*]. The > user should depend upon this. [*] This is not entirely true: 1) If the Python compiler is run with the -O switch, then the bytecodes for the type-assert are NOT generated. This is analogous to the behavior for the "assert" statement today. 2) If the Python runtime is run with the -O switch, then any bytecodes generated for an assert will not be executed. Again: analogous to the current "assert" statement today. Background: Given: assert expression, message Python generates: if __debug__ and not expression: raise AssertionError, message The above code is not generated when -O is present. __debug__ is false when -O is present. ------------------------------- This is a small part of the definition of the type-assert operator's behavior at bytecode-generation and at run-time. It does not affect its behavior at type-check-time. Cheers, -g -- Greg Stein, http://www.lyra.org/ From gstein@lyra.org Wed Jan 5 23:43:56 2000 From: gstein@lyra.org (Greg Stein) Date: Wed, 5 Jan 2000 15:43:56 -0800 (PST) Subject: [Types-sig] updated proposal (fwd) In-Reply-To: <3873494E.F78699C@maxtal.com.au> Message-ID: On Thu, 6 Jan 2000, skaller wrote: >... > More precisely: b is a string from the point at which b is > bound to the checked expression, at least until: > > 1) 'end of block' > 2) An exec statement is seen > 3) An assignment to b > 4) If b is global, then a function call > 5) the module dictionary is fiddled with hehe... then I guess you haven't seen code like this: def f(): a = 5 g() print a def g(): try: raise 'hi' except: t, v, tb = sys.exc_info() print tb.tb_frame.f_back.f_locals['a'] f() Due to some sneaky behavior in Python, you can't actually change the locals through that dictionary. Python keeps a separate "fast locals" structure. There are cases where Python moves locals back and forth between the dictionary and the fast locals, but I don't recall those rules. But there may be a way for another function to affect a function's locals... hehe. Of course, we don't have to worry about it because I can bet you that Guido would be completely opposed to programs doing that :-) [ actually, there should be a way, so that Python debuggers can alter locals... ] >... rest of note ... I agree completely. Well-stated. Cheers, -g -- Greg Stein, http://www.lyra.org/ From tim_one@email.msn.com Thu Jan 6 01:30:30 2000 From: tim_one@email.msn.com (Tim Peters) Date: Wed, 5 Jan 2000 20:30:30 -0500 Subject: [Types-sig] type-asserts and check/run -time (was: updated proposal) In-Reply-To: Message-ID: <000901bf57e5$9e866b80$b1a0143f@tim> [Greg Stein] > ... > You and I know what that code says. Why should we state that > the type-checker can never be as smart as us? The whole > *point* of the type-checker is for it to be smarter than we > are. We are trying to use it to find the places where we are > being stupid. Actually, that's what you're trying to use it for. I maintain that most people who want static typing are going to wag their heads in bewilderment at anything fancier (or muddier) than a traditional "declare it or die" system of purely compile-time annotations. An interesting contrast is the recent Mercury language, basically Prolog on hyperspeed steroids: http://munkora.cs.mu.oz.au/research/mercury/index.html Their inferencers (plural because Prolog has stuff beyond types that needs to be guessed -- e.g., whether a clause "generates" exactly one, no more than one, or any number of results) are quite capable, but are used only to verify the user's explicit declarations. Everything must be declared. Prolog is such that it's not hard to construct programs for which inference is impossible; Mercury's response in such cases is to reject the program at compile-time (i.e., if they can't prove your declarations are correct, you lose, case closed, no appeal to the slower runtime court). Mercury's concerns are industrial-strength safety and industrial-strength speed, and their rigid declaration rules are what that crowd expects: they don't want "help", they want absolute compile-time guarantees. Anyway, I'm not fighting this one way or the other anymore -- if you're writing the code, you win . I'll whine about it later -- provided there's actually whineworthy behavior. more-code-less-talk-ly y'rs - tim From tim_one@email.msn.com Thu Jan 6 02:04:48 2000 From: tim_one@email.msn.com (Tim Peters) Date: Wed, 5 Jan 2000 21:04:48 -0500 Subject: [Types-sig] OPINIONS: Warnings In-Reply-To: <38723F11.B5939730@prescod.net> Message-ID: <000101bf57ea$68cd1de0$0aa0143f@tim> [Paul Prescod] > This is an important issue that I would like a lot of feedback > about. I will state my opinion and invite Greg and others to > state theirs. > > It is my opinion that we should not, in our specification, flag > any dubious constructs as being constructs that the compiler > should warn about. Anything dubious should be merely disallowed. > Warnings are for getting around language flaws. Also for pointing out less than perfect compile-time knowledge; i.e., I think it depends on the "dubious construct". For example, if the scheme we come up with here cannot, in some cases, guarantee to resolve a type question at compile-time, some people aren't going to care, extreme safety- and/or speed-freaks are going to want those treated as compile-time errors, and "everyone else" is going to want to see compile-time warnings. > ... > IMO, we will never be able to totally get rid of warning > and "lints" but we should acknowledge them as byproducts > of imperfect designs created by imperfect humans. Or of best-possible designs facing intractable problems. > If we design any part of our system *expecting* a > warning to be issued then we are knowingly designing > in flaws. Or simply acknowledging that the whole problem can't be solved at compile-time -- for some definitions of "the whole problem". warning-function-not-inlined-ly y'rs - tim From paul@prescod.net Thu Jan 6 03:13:10 2000 From: paul@prescod.net (Paul Prescod) Date: Wed, 05 Jan 2000 22:13:10 -0500 Subject: [Types-sig] compile/run -time (was: syntax and compile/run -time) References: Message-ID: <3873DE16.AFC7769E@prescod.net> Greg Stein wrote: > > The "runtime" keyword presumes too much knowledge on the programmer's > part. How are they supposed to know the full bounds and limitations of the > type checker? If you say, "you must insert 'runtime' in places where the > type checker can't figure out something at compile-time," then they're > next question will be, "okay. when is that?" That's *exactly* my point. Now we are getting to the nub of the problem. I know exactly what the Java type checker will and will not check at compile time. The rules are simple enough that I can fit them all in my head. If we develop a type system that does not allow a typical programmer to fit all of the rules in their head then we have, in my opinion, failed. > Instead, allow all the forms and make the type-checker do its best. The > code will always work properly. "Doing its best" is not good enough. I'm turning on type checking because I want it to do its job completely and properly and give me a big fat error message if it cannot. > If somebody marks a function as "typesafe", then the checker should raise > an error if it cannot determine a particular typedecl condition (like with > the "a=get_an_interface()" example). If a function is not marked as > "typesafe", then we will not have lost any functionality over 1.5, but we > certainly have gained: the argument will be checked on function entry. The problem is that the semantic distance between the place where they made the mistake and the place where they run into a problem could be quite long. If I write a typesafe function that depends on an interface that depends on an interface, that depends on an interface, that depends on an interrace that uses runtime types, I have a lot of backtracking to do. Instead, the "runtime" keyword should be contagious so that as soon as you use it, you KNOW. You can't use runtime types without knowing that you are doing so because you are going to run into an error message sooner or later. Therefore requiring the user to understand the issue sooner rather than later is doing them a favor. > a = get_an_interface() > def foo(x: a): > ... > > That is much better than what I can do today. No, not really: a = get_an_interface() def foo(x): if not a.check( x ): raise TypeError If it can't be checked at compile time, it's syntactic sugar for stuff we can do already. > It requires the programmer to know too much about the type-checker. I > think we have the runtime/compile-time issue covered without this. Programmers should know all about type checkers. That's their job. It's our job to make things simple enough so that they can do their job. > a = typedef Int or String > > is always evaluated at runtime. It creates a typedecl object and assigns > that to "a". The question is whether the type-checker can do anything with > it at compile-time. In most cases: yes, it can understand the situation. > In atypical cases: no, it can't and bugger on those people. I don't see the type checker as a side effect that sometimes works and sometimes doesn't. The type checker is the main event. It's what we are here to design. New runtime features are the side effect. -- Paul Prescod - ISOGEN Consulting Engineer speaking for himself The occasional act of disrespect for the American flag creates but a flickering insult to the values of democracy -- unless it provokes America into limiting the freedoms that are its hallmark. -- Paul Tash, executive editor of the St. Petersburg Times From scott@chronis.pobox.com Thu Jan 6 07:58:59 2000 From: scott@chronis.pobox.com (scott) Date: Thu, 6 Jan 2000 02:58:59 -0500 Subject: [scott@chronis.pobox.com: Re: [Types-sig] type-asserts and check/run -time (was: updated proposal)] Message-ID: <20000106025859.A8438@chronis.pobox.com> On Wed, Jan 05, 2000 at 03:07:19PM -0800, Greg Stein wrote: > On Wed, 5 Jan 2000, scott wrote: > > On Thu, Jan 06, 2000 at 12:38:22AM +1100, skaller wrote: > > > > Given the following code: > > > > > > > > decl a: Int > > > > b = a ! String > > > > > > > > The compiler will know that it will always fail. So it can flag it. In the > > > > following: > > > > > > > > decl a: Any > > > > b = a ! String > > > > > > > > It has no idea. So it lets it pass, to be evaluated at runtime. The > > > > operator *is* a runtime operator. The reason it is important to the > > > > compile-time checker, however, is that the checker now knows that "b" is a > > > > String. It cannot be anything else (or the exception would prevent the > > > > assignment). > > > > OK, what I see as wishy-washy about the behavior of the thing is that > > it does one thing at compile time and another at run time. As skaller > > *everything* is evaluated differently at runtime and compile-time. > Everything. As you described on your web page, the type-assert operator works with the type system in two ways. at compile time it flags one set of problems; at run time, another. I would prefer it to only assert type in one way at one time. No "at compile time it checks foo, and at run time it checks bar". The difference is distinct and more pointed than the fact that everything is different in a type system at run time vs compile time. The semantic changes, and it is unduly confusing. [...] > > Yes, it is complex. So? There is nothing unique about this operator. > Consider: There is something unique, described above, and detailed below. > > a = 5 > if f(): > a = "5" > b = a > > In the above construct, we have to consider the same five conditions that > skaller noted. There is nothing special/unique about the type-assert > operator. > > My point here is that the checker must be considering those five *all* the > time, for *all* operators, statements, and control structures. > > This is where Tim Peters will pipe in and say "but if everything is > required to be declared, then the problem is reduce and those five > conditions to not necessarily (all) apply." We can require a declaration > for every single variable that you ever want to use, or we can add some > smarts to the checker. Your smarts are my mud. But the alternative is *not* restricted to requiring a declaration for every single variable. There is atleast one other alternative which would be IMO very smart and much less muddy. I have tried to explain this alternative 2 times before, but I must not be expressing myself well... let me try again: The goal: At the end of compile time, we want _all_ of our _explicit_ type declarations verified 100%. The only possible exception is when we explicitly state in our type declarations that we want the declaration to be deferred until runtime (no-op at compile time). If our system of explicit type declarations are unverifiable at compile time, the compile time type checker flags an error condition. The only exception to this rule should be when walking a parse tree, we refuse to compile a dynamically generated one. For all deduced (not explicitly declared, or declared to be runtime dependant) types, the compile time system does the best it can, and guesses by means of the type-unioning mechanism you have started to build. Here the developer _knows_ she is playing games with a system that might not act as expected in the face of dynamicism and _knows_ when she is playing with a system that is more absolute. The type checker may either warn about fishy things for deduced types or it may simply allow them. How this can be accomplished: 1) For each typedeclarator object, record whether or not it is a deduced type (defined by it being not explicitly declared or declared specifically for runtime use if we hold onto that concept) 2) For all _deduced_ types, use the unioning behavior you have already started to build. It seems to work well for that. 3) for all _explicit_ types, redefine the 'or' operator in type declarations so that it behaves as follows. ------------------case 1: # # valid code -- b is _deduced_ and used liberally # decl f def(x: String) -> None decl g def(x: Int) -> None decl a: Int a = 5 b = a g(b) b = "s" f(b) -----------------case 2: # # valid code with a different semantic for the 'or' type declaration # operator. Works with your prototype much like it would with the # differnent semantic for 'or'. This should be ok. # decl f def(x: Int or None) -> Int decl g def(x: String) -> Int or None def f(x): if x is None: return 0 else: return x + 1 def g(x): if len(x): return len(x) else: return None f(g("")) f(g("string with length")) -----------------case 3: # # invalid code with different semantic for the 'or' operator in type # declarations. Not sure what your type-unioning would do with this # but it seems like it would maybe issue a warning and let it go until # runtime. With new semantic, this is an error at compile time and # won't run with type checking turned on. # decl f def(x: Int) -> Int decl g def(x: String) -> Int or None def f(x): return x + 1 def g(x): if len(x): return len(x) else: return None f(g("")) # error, types don't match: f expects Int, g returns Int or String f(g("string with length")) # error, types don't match: f expects Int, g returns Int or String ------------------case 4: # # more invalid code. under your current type-unioning would issue warning # and then run. under this semantic for 'or', it's an error at compile time, # no questions asked. It won't run when static typing is turned on. # decl f def(x: Int) -> String decl g def(x: String) -> Int or None decl a: Int or None def f(x): return "%02d" % (x + 1) def g(x): if len(x) > 3: return len(x) else: return None # we can assign a to be of a type dependent on run time evaluation # so long as that type is in the set of or'd types. a = g(sys.argv[1]) # but when we don't know whether a is an int for sure, # we don't defer checking for run time assertions. instead, # it is a compile time error, period, and won't even run # when type checking is turned on. assert type(a) == type(0) # (or use 'a ! 0' for syntactic sugar.) f(a) ------------------- more on case 4: In this case, we don't know at compile-time that a is an int when f(a) is executed. Instead, all we know is that a is either an int, or an exception is raised at run time when optimization is not active. It's not type safe unless we never declare a to begin with, in which case we would treat it like the variable 'b' in case 1, and that would be OK. > I prefer the latter, and put forward my prototype > as an example that much can be done already (with no declarations). Your prototype rocks, it's much more than I've had time to even think about. We should use it as much as we are able to come to a consensus on it. If there's not a consensus, then it's not 'the python static typing system' and we need make just that. > > > To me, if the static typing information states that a is 'Any', then > > if at any other point, a is considered a string, then the static > > typing error is flagged because it is not type-safe in terms of the > > presented static typing information. > > String is a subtype of Any, so there shouldn't be an error. If the checker > wants to make a finer-grained assessment of "a", then it should be able > to. Given: I've put together a better description of the issue above. that's try 3. If it still doesn't make sense, we need to find a better way to talk about it. The issue is not whether String is a subtype of any, it's whether you can use assertions to defer the type checking till runtime. > > In other words, the static type > > information should be a closed system. It's *much* easier to work > > with that way -- both in terms of implementing it and using it. > > I don't understand "closed system" in this context. At the end of compile time, it's 100% verifiable in terms of all explicitly declared type information holding true for the program. If it's not verifiable, and we must defer the type checking to run time using assertions for explicityly declared types, it's not closed at compile time. > > > I don't think this should be deferred till run time by the static type > > checker. Instead, it seems to me that: > > > > decl a: Any > > decl def f(a: String) -> Whatever > > a ! String > > f(a) > > > > is not type safe... especially given the complexity of understanding > > the implications of `!' at compile-time. If it is indeed essentially > > a run time operator, then it should not affect the compile-time > > checking. > > The type-checker as I've been coding it (so far) would not remap "a" to a > String. The checker sees: > > ! String > > Given that: the f(a) will raise an error. > > A smart type-checker *would* remap "a" for some period and allow the above > code to be compiled without flagging an error. And why shouldn't it? because we want something that provably works with minimal (and simple) outlandish cases, all at compile time. > > You and I know what that code says. Why should we state that the > type-checker can never be as smart as us? The whole *point* of the > type-checker is for it to be smarter than we are. We are trying to use it > to find the places where we are being stupid. The whole point of the type checker is to produce a coherent system that is as simple as possible for the developer to mentally navigate. If we make it 'smart', it's will confuse a lot more people than if we make it simple for the developer to mentally navigate and does it's job 100% at compile time. > > >... > > So, when I stated before that it would be nice if we could take the > > type-assert operator and make it flag all errors at compile time, even > > if it meant following a few simple restrictions to gain that feature, > > I'd like to re-phrase that as > > > > the type-assert operator needs to either work completely at compile > > time, or completely at run time. It seems dubious that we can work > > out a simple enough system to use the thing at compile time, though it > > might be possible. As a run-time only tool, it doesn't seem to offer > > substantially more than asserts do now. > > The type-assert operator is defined to *always* operate at runtime[*]. The > user should depend upon this. > > My statement was that a type-checker can detect certain situations: > > decl a: Any > a = 5 > b = a ! String > > Given the above code, I *want* the type-checker to raise an error. I was > being an idiot. I assigned an integer, then expected a string. > > And you are correct: as a runtime tool, it *is* nothing more than an > assert. Its reason-to-exist is because the type-checker can gather a LOT > of information from it. It's this dual nature of the thing that makes a lot very confusing. Also, it weakens the potential strength of the type checker as shown above. Also, it just seems wrong to me to have an syntactic construct that is "essentially" a runtime thing, whose "reason-to-exist" is to provide information to the compile time type checker. There are too many contradictions in that. > > Its definition of operation also lets us leverage what happens with > function arguments: > > def foo(x: a): > some_code() > > can be written as: > > def foo(x): > x ! a > some_code() > > It is simply that the former is easier to write AND it provides a > signature for callers to use. It also weakens the ability of compile time checking to approach 100% verifiability. > > > Furthermore, the tools provided to work with typing information at > > run-time should in no way alter the behavior of the compile-time > > system. > > I'm not sure if my above explanations satisfy this. I see the two as > distinct operations, with no dependencies between them. You stated a dependency for the type-assert operator above: the type checker can glean a lot of information from it. I don't think it should glean that information for explicitly declared types, only for deduced ones. because in the case of explicitly declared types, you are deferring problems to run time, very possibly without intending to do so. > > The type-checker derives information from what it knows about runtime > operation. That is a given. I also believe the bytecodes that are > generated will always be the same, despite what the type-checker says > about things. Yes, but from the type assert operator, it doesn't know the type, it only knows that either the type will hold, or an exception will be raised at run time when optimization is not turned on. It should know that the types _will_ hold at compile time, and when activated, refuse to run the code in the first place unless the types are verified. scott From scott@chronis.pobox.com Thu Jan 6 08:52:08 2000 From: scott@chronis.pobox.com (scott) Date: Thu, 6 Jan 2000 03:52:08 -0500 Subject: [Types-sig] updated proposal (fwd)] In-Reply-To: References: <20000105151534.A4782@chronis.pobox.com> Message-ID: <20000106035208.C8723@chronis.pobox.com> On Wed, Jan 05, 2000 at 01:53:41PM -0800, Greg Stein wrote: > > > > regarding a new namespace to place interface/type info into... I'd > > > > suggest leaving that open for the time being, or just defining that > > > > access is available at runtime via a function call such as: > > > > > > > > interfaces() -> [ all known interfaces in current namespace ] > > > > interfaces(x: Any) -> [ all known interfaces that x implements ] > > > > > > > > this would allow 'implementors' to put the darn info wherever they > > > > want and define a flexible enough interface to getting the interfaces > > > > to allow for variable underlying implementations. (sorry for > > > > mouthful...) > > > > > > Interesting thought, but I feel confident that we don't need another > > > namespace, so I don't really see a reason to make the above compromise. > > > > how would it compromise anything about where type info is stored? > > Sorry. I didn't mean that the info would be compromised... I meant that I > would not compromise my position :-) In other words, I'm feeling > righteous enough :-) that I think we don't need a new namespace or an > interfaces() function for this problem. Are you feeling righteous enough that you think wherever we store the info now is where it'll be for next millenium? It seems safer to take this approach even if it is unnecessary at the moment. scott From gstein@lyra.org Thu Jan 6 11:48:28 2000 From: gstein@lyra.org (Greg Stein) Date: Thu, 6 Jan 2000 03:48:28 -0800 (PST) Subject: [Types-sig] updated proposal (fwd)] In-Reply-To: <20000106035208.C8723@chronis.pobox.com> Message-ID: On Thu, 6 Jan 2000, scott wrote: >... > Are you feeling righteous enough that you think wherever we store the > info now is where it'll be for next millenium? It seems safer to take > this approach even if it is unnecessary at the moment. sorry, but... Yup :-) hehe. couldn't resist, especially after a good night of several drinks :-) Seriously, though... Yah. I am thinking of the long term. I get rather peeved with the things that I feel are overly restrictive and not well-integrated with the current Python model. I want to see extensions, rather than a new, partitioned set of functionality. The discussions on this SIG have been extremely enlightening. I have definitely had to re-evaluate and reconsider some of my original thoughts. There have been a number of things that I thought we shouldn't do, or that were not Good And Right. I've tempered a number of my thoughts there, but also firmed up a number of beliefs. Writing a prototype has also been enlightening. I feel much more comfortable with some of my recommendations -- what I "believed" was workable, I've reduced to "done." [no, the tool doesn't do what our vision of a type-checker does, but it has validated (for me) some of my thinking, which I've found to be useful] Back to your question: if we were to design for the next *millenium*, then we would not get anywhere. Hell... even worse, we'd end up with Ada. I'm not settled with the parameterization stuff because I haven't come up with a syntax that I feel is natural and Pythonic. But the rest: hell yah. It feels good, complete, and clean. My problem is that I doubt that I'm explaining the "whole" effectively. The "whole" has snapped into place, after the past several weeks of discussion and refinement. I see a path, so I started to code a type-checker for that path. It is a prototype and demo, to help explore some of the details and edge cases, but also to help me to explain my thoughts ("go look at the code!"). Bah. I shouldn't write these things after drinking... but it *is* awfully fun to do so... hehe... Cheers, -g -- Greg Stein, http://www.lyra.org/ From skaller@maxtal.com.au Thu Jan 6 12:02:03 2000 From: skaller@maxtal.com.au (skaller) Date: Thu, 06 Jan 2000 23:02:03 +1100 Subject: [Types-sig] type-asserts and check/run -time (was: updated proposal) References: <000901bf57e5$9e866b80$b1a0143f@tim> Message-ID: <3874843B.3D0671FA@maxtal.com.au> Tim Peters wrote: > An interesting contrast is the recent Mercury language, basically Prolog on > hyperspeed steroids: > > http://munkora.cs.mu.oz.au/research/mercury/index.html Hmm .. not so sure Fergus would like that description :-) -- John (Max) Skaller, mailto:skaller@maxtal.com.au 10/1 Toxteth Rd Glebe NSW 2037 Australia homepage: http://www.maxtal.com.au/~skaller voice: 61-2-9660-0850 From skaller@maxtal.com.au Thu Jan 6 12:15:59 2000 From: skaller@maxtal.com.au (skaller) Date: Thu, 06 Jan 2000 23:15:59 +1100 Subject: [Types-sig] updated proposal (fwd) References: Message-ID: <3874877F.716808C1@maxtal.com.au> Greg Stein wrote: > > C/C++ has distinct namespaces for certain names. Not really. C has a separate space for structure names, but C++ has a unified space, plus some hacks and kludges for compatibility with C. > hmm, i see the complication, and could live with this either way, but > I'm not convinced that there does not exist the possibilty of a way to > make interfaces and classes share names that is reasonable, just that > it's unlikely that one exists. thanks for clarifying. class X: ... x = X() I'd say that x, not X, has interface X, if anything does. -- John (Max) Skaller, mailto:skaller@maxtal.com.au 10/1 Toxteth Rd Glebe NSW 2037 Australia homepage: http://www.maxtal.com.au/~skaller voice: 61-2-9660-0850 From skaller@maxtal.com.au Thu Jan 6 12:46:26 2000 From: skaller@maxtal.com.au (skaller) Date: Thu, 06 Jan 2000 23:46:26 +1100 Subject: [Types-sig] updated proposal (fwd) References: <3873494E.F78699C@maxtal.com.au> <20000105144839.B4237@chronis.pobox.com> <20000105154254.B4782@chronis.pobox.com> Message-ID: <38748EA2.C6B35115@maxtal.com.au> scott wrote: > If I declare somevar as having type 'Any' it means that it is the Any > type (which in itself doesn't enforce anything other than the variable > cannot be used in a way that asserts a particular type). When > something is not declared, it can be deduced from a relatively > predictable algorithm, but that deduced type is one which allows the > more permissive 'one of a or b or ... or n'. Declared types seem more > useful as 'always any of a or b or ... or n' to me. If we wanted the > 'one of...' behavior in type declarations, I guess I'd be more inclined to > call that 'xor' than 'or'. Given a choice between the two, I'd prefer > the 'always any of...' semantics. The question is not a matter of your personal choice but logic/mathematics. Consider a function: f : Int or String -> Int or String The function is correct only if it accepts MORE inputs than the static declaration. For the return, it is the other way around: the function must return LESS than the specified types: for example, always an Int would be OK. But returning a float would not be. So, when type checking the FUNCTION, the interpretation of the static type declarator for the argument and the return value is different. [In fact, dual: the comparison is reversed] On the other hand, checking a _call_, as opposed to a function: what can be passed in is LESS than the specification. But the client can apply a procedure to the return value that works for MORE types than the specification. My point: the 'behaviour' of a static interface declaration depends on context. There is no single interpretation. -- John (Max) Skaller, mailto:skaller@maxtal.com.au 10/1 Toxteth Rd Glebe NSW 2037 Australia homepage: http://www.maxtal.com.au/~skaller voice: 61-2-9660-0850 From skaller@maxtal.com.au Thu Jan 6 12:49:40 2000 From: skaller@maxtal.com.au (skaller) Date: Thu, 06 Jan 2000 23:49:40 +1100 Subject: [Types-sig] updated proposal (fwd) References: Message-ID: <38748F64.61475E65@maxtal.com.au> Greg Stein wrote: > > On Thu, 6 Jan 2000, skaller wrote: > >... > > More precisely: b is a string from the point at which b is > > bound to the checked expression, at least until: > > > > 1) 'end of block' > > 2) An exec statement is seen > > 3) An assignment to b > > 4) If b is global, then a function call > > 5) the module dictionary is fiddled with > > hehe... then I guess you haven't seen code like this: > > Due to some sneaky behavior in Python, you can't actually change the > locals through that dictionary. i know: that's why the conditions say 'if b is global', and 'the module dictionary', both cases excluding locals. -- John (Max) Skaller, mailto:skaller@maxtal.com.au 10/1 Toxteth Rd Glebe NSW 2037 Australia homepage: http://www.maxtal.com.au/~skaller voice: 61-2-9660-0850 From scott@chronis.pobox.com Thu Jan 6 18:28:48 2000 From: scott@chronis.pobox.com (scott) Date: Thu, 6 Jan 2000 13:28:48 -0500 Subject: [Types-sig] updated proposal (fwd) In-Reply-To: <38748EA2.C6B35115@maxtal.com.au> References: <3873494E.F78699C@maxtal.com.au> <20000105144839.B4237@chronis.pobox.com> <20000105154254.B4782@chronis.pobox.com> <38748EA2.C6B35115@maxtal.com.au> Message-ID: <20000106132848.A11233@chronis.pobox.com> On Thu, Jan 06, 2000 at 11:46:26PM +1100, skaller wrote: > scott wrote: > > > If I declare somevar as having type 'Any' it means that it is the Any > > type (which in itself doesn't enforce anything other than the variable > > cannot be used in a way that asserts a particular type). When > > something is not declared, it can be deduced from a relatively > > predictable algorithm, but that deduced type is one which allows the > > more permissive 'one of a or b or ... or n'. Declared types seem more > > useful as 'always any of a or b or ... or n' to me. If we wanted the > > 'one of...' behavior in type declarations, I guess I'd be more inclined to > > call that 'xor' than 'or'. Given a choice between the two, I'd prefer > > the 'always any of...' semantics. > > The question is not a matter of your personal choice > but logic/mathematics. > > Consider a function: > > f : Int or String -> Int or String > > > The function is correct only if it accepts MORE inputs > than the static declaration. > > For the return, it is the other way around: the function > must return LESS than the specified types: for example, > always an Int would be OK. But returning a float > would not be. > > So, when type checking the FUNCTION, the interpretation > of the static type declarator for the argument and > the return value is different. [In fact, dual: the comparison > is reversed] > > On the other hand, checking a _call_, > as opposed to a function: what can be passed in is LESS > than the specification. But the client can apply a procedure > to the return value that works for MORE types than the specification. > > My point: the 'behaviour' of a static interface declaration > depends on context. There is no single interpretation. I see this now, and presented what I was trying to get at in a subsequent post much more along these lines. Yes, it depends on context as you state. There's an important piece of context that works in greg's web page (and the last version of the code I looked at) which causes that type system to defer a whole class of checks until run time. The context is within code blocks where variables are found to be LESS by means of a type-assertion. While this is great when applied to variables which have no explicit type declaration associated with them, it weakens the position of the system at the end of compile time when it is applied to explicitly declared types. scott From tim_one@email.msn.com Fri Jan 7 01:19:04 2000 From: tim_one@email.msn.com (Tim Peters) Date: Thu, 6 Jan 2000 20:19:04 -0500 Subject: [Types-sig] type-asserts and check/run -time (was: updated proposal) In-Reply-To: <3874843B.3D0671FA@maxtal.com.au> Message-ID: <000601bf58ad$3077b040$932d153f@tim> [Tim] > An interesting contrast is the recent Mercury language, > basically Prolog on hyperspeed steroids: [John Skaller] > Hmm .. not so sure Fergus would like that description :-) That's the eternal conflict between research and marketing -- if he doesn't like it, he should change the web page. wondering-whether-they-even-let-him-near-it-ly y'rs - tim From Tony Lownds Sat Jan 8 13:40:07 2000 From: Tony Lownds (Tony Lownds) Date: Sat, 8 Jan 2000 05:40:07 -0800 (PST) Subject: [Types-sig] run-time typer (was Re: syntax and compile/run -time) In-Reply-To: <200001052033.PAA19771@p5mts.nielsenmedia.com> Message-ID: On Wed, 5 Jan 2000 mengx@nielsenmedia.com wrote: > If using docstring or a special varible name like "types = {a:"int"}" > are of some trouble somehow, could it be possible to use the following format ? > > class aClass: > pass > > def foo_type_string(a,b_type_int, c_type_file,d_type_aClass): > > return c.read() Hey, that could be useful.... no need to hack Grammar files... I have made a usable prototype run-time checker that does this: http://tony.lownds.com/python/typesys.py exploring all the options :) -Tony From tony@metanet.com Sun Jan 9 01:45:58 2000 From: tony@metanet.com (Tony Lownds) Date: Sat, 8 Jan 2000 17:45:58 -0800 (PST) Subject: [Types-sig] correct url: http://tony.lownds.com/python/typefn.py In-Reply-To: Message-ID: oops! sorry. -Tony On Sat, 8 Jan 2000, Tony Lownds wrote: > > On Wed, 5 Jan 2000 mengx@nielsenmedia.com wrote: > > > If using docstring or a special varible name like "types = {a:"int"}" > > are of some trouble somehow, could it be possible to use the following format ? > > > > class aClass: > > pass > > > > def foo_type_string(a,b_type_int, c_type_file,d_type_aClass): > > > > return c.read() > > > Hey, that could be useful.... no need to hack Grammar files... > I have made a usable prototype run-time checker that does this: > > http://tony.lownds.com/python/typesys.py > > exploring all the options :) > > -Tony > > > _______________________________________________ > Types-SIG mailing list > Types-SIG@python.org > http://www.python.org/mailman/listinfo/types-sig > From mengx@nielsenmedia.com Sun Jan 9 18:40:56 2000 From: mengx@nielsenmedia.com (mengx@nielsenmedia.com) Date: Sun, 9 Jan 2000 13:40:56 -0500 (EST) Subject: [Types-sig] Re: correct url: http://tony.lownds.com/python/typefn.py Message-ID: <200001091840.NAA09465@p5mts.nielsenmedia.com> Just another thought, would it possible for some sort of function overloading. def foo_type_string(a_type_string): pass def foo_type_int(a_type_int): pass a=1 foo(a) # -> foo_type_int a='1' foo(a) # -> foo_type_string I am not sure the side effect of it. -Ted Meng > From POP3 Sat Jan 8 20:49:37 2000 > Date: Sat, 8 Jan 2000 17:45:58 -0800 (PST) > From: Tony Lownds > X-Sender: tony@adam12 > To: types-sig@python.org > cc: mengx@nielsenmedia.com > Subject: correct url: http://tony.lownds.com/python/typefn.py > MIME-Version: 1.0 > > > oops! sorry. > > -Tony > > On Sat, 8 Jan 2000, Tony Lownds wrote: > > > > > On Wed, 5 Jan 2000 mengx@nielsenmedia.com wrote: > > > > > If using docstring or a special varible name like "types = {a:"int"}" > > > are of some trouble somehow, could it be possible to use the following format ? > > > > > > class aClass: > > > pass > > > > > > def foo_type_string(a,b_type_int, c_type_file,d_type_aClass): > > > > > > return c.read() > > > > > > Hey, that could be useful.... no need to hack Grammar files... > > I have made a usable prototype run-time checker that does this: > > > > http://tony.lownds.com/python/typesys.py > > > > exploring all the options :) > > > > -Tony > > > > > > _______________________________________________ > > Types-SIG mailing list > > Types-SIG@python.org > > http://www.python.org/mailman/listinfo/types-sig > > > From mwh21@cam.ac.uk Mon Jan 10 20:48:42 2000 From: mwh21@cam.ac.uk (Michael Hudson) Date: Mon, 10 Jan 2000 20:48:42 +0000 (GMT) Subject: [Types-sig] Last month's activity Message-ID: Could some kind person email me last month's messages from the Types-SIG? I had to delete them to avoid going over my mail quota during the xmas break :-( and the archives on python.org don't have enough headers for gnus to thread :-( So if someone can email me this (gzipped standard mbox format if possible), can you drop me a note? (I don't need ten copies of it!) Thanking you very much in advance, Michael From gstein@lyra.org Tue Jan 11 10:31:15 2000 From: gstein@lyra.org (Greg Stein) Date: Tue, 11 Jan 2000 02:31:15 -0800 (PST) Subject: [Types-sig] run-time typer In-Reply-To: Message-ID: On Sat, 8 Jan 2000, Tony Lownds wrote: >... > Hey, that could be useful.... no need to hack Grammar files... > I have made a usable prototype run-time checker that does this: > > http://tony.lownds.com/python/typesys.py > > exploring all the options :) Interesting... :-) But really -- we do need a static checker. I'm not sure how this helps in that direction. Certainly, it can work as an interim solution! Cheers, -g -- Greg Stein, http://www.lyra.org/ From gstein@lyra.org Tue Jan 11 11:53:17 2000 From: gstein@lyra.org (Greg Stein) Date: Tue, 11 Jan 2000 03:53:17 -0800 (PST) Subject: [Types-sig] updated proposal again Message-ID: Hi all, I've updated my type proposal again: * detailed incomplete classes and interfaces * detailed/refined the "decl" statement * detailed the type-assert operator The document is at: http://www.lyra.org/greg/python/typesys/type-proposal.html You can see a diff at: http://www.lyra.org/cgi-bin/viewcvs.cgi/gjspy/typesys/type-proposal.html.diff?r1=1.3&r2=1.4 Cheers, -g p.s. Tony and other grammar hackers: look at the new 'decl' statement. I added a little helper grammar rule there. -- Greg Stein, http://www.lyra.org/ From tismer@tismer.com Tue Jan 11 12:42:06 2000 From: tismer@tismer.com (Christian Tismer) Date: Tue, 11 Jan 2000 13:42:06 +0100 Subject: [Types-sig] Last month's activity References: Message-ID: <387B251E.25CE31B5@tismer.com> Michael Hudson wrote: > > Could some kind person email me last month's messages from the Types-SIG? > I had to delete them to avoid going over my mail quota during the xmas > break :-( > > and the archives on python.org don't have enough headers for gnus to > thread :-( > > So if someone can email me this (gzipped standard mbox format if > possible), can you drop me a note? (I don't need ten copies of it!) I've sent him a copy, so this is finished I think. ciao - chris -- Christian Tismer :^) Applied Biometrics GmbH : Have a break! Take a ride on Python's Düppelstr. 31 : *Starship* http://starship.python.net 12163 Berlin : PGP key -> http://wwwkeys.pgp.net PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF we're tired of banana software - shipped green, ripens at home From mwh21@cam.ac.uk Tue Jan 11 23:38:15 2000 From: mwh21@cam.ac.uk (Michael Hudson) Date: 11 Jan 2000 23:38:15 +0000 Subject: [Types-sig] Last month's activity In-Reply-To: Christian Tismer's message of "Tue, 11 Jan 2000 13:42:06 +0100" References: <387B251E.25CE31B5@tismer.com> Message-ID: Christian Tismer writes: > Michael Hudson wrote: > > > > Could some kind person email me last month's messages from the Types-SIG? > > I had to delete them to avoid going over my mail quota during the xmas > > break :-( > > > > and the archives on python.org don't have enough headers for gnus to > > thread :-( > > > > So if someone can email me this (gzipped standard mbox format if > > possible), can you drop me a note? (I don't need ten copies of it!) > > I've sent him a copy, so this is finished I think. > Now I've *finally* got gnus behaving itself, thank you very. I think you might have missed a couple articles right at the start, but now I have the chance to read all that lovely discussion again, and threaded this time. I hope this helps. Thanks again, Michael From tony@metanet.com Thu Jan 13 03:17:56 2000 From: tony@metanet.com (Tony Lownds) Date: Wed, 12 Jan 2000 19:17:56 -0800 (PST) Subject: [Types-sig] run-time typer In-Reply-To: Message-ID: > > Interesting... :-) > > But really -- we do need a static checker. I'm not sure how this helps in > that direction. Certainly, it can work as an interim solution! > Actually I wouldn't use this technique in production code at all because it would slow function calls a lot. So, I was thinking of examining the bytecode to see if a function that used the run-time type checks actually returns the correct type, and checking uses of other rttc'd function from within the function, and then simply removing the checks. But I havent had time to code that. Also Tim Meng's follow-up idea made me realize that people will want to write code differently and therefore want this type system to work in different ways. I agree, a run-time checker is not the goal here. -Tony From tony@metanet.com Thu Jan 13 04:45:18 2000 From: tony@metanet.com (Tony Lownds) Date: Wed, 12 Jan 2000 20:45:18 -0800 (PST) Subject: [Types-sig] updated proposal again In-Reply-To: Message-ID: My grammar hacks no longer works for some reason, its building python executables that core-dump. I'll have to figure this out some weekend soon... Regarding decl: Adding that intermediate rule adds more keywords, you should update the top of your proposal to reflect these. "Only two new keywords: typedef and decl. Possibly any. One new operator: !. One new lexical token: ->." How do you declare that a local variable is a certain type? This seems to be a purposeful inability; perhaps you can elaborate why that is on the page. Or, you could add "decl var x:Int" as a valid syntax. Regarding !: How about using normal functions to do type-asserts and declaring that these functions can be skipped if the type machinery knows the argument is the same type as the return type: decl assert foo : def(Any) -> sometype def foo(x): if type(x) == SomeType: return x raise TypeAssertionError, (x, SomeType) Reasons in favor: 1. Lots of built-in functions could use this model: list, int, str, tuple, float, complex, long, etc. 2. Makes the code written using the type system more compatible with standard Python installations. 3. Gives the user more control over how a type assert is implemented. For complicated types, especially parameterized types, it may not be clear what algorithm a type assert uses. I wouldn't like a hidden O(n^2) in code just for using a ! operator... n = 1000000 y = [[1]*n]*n # ... operations on y that confuse the type checker ... y = y ! List(Int or List(Int)) -Tony > > -- > Greg Stein, http://www.lyra.org/ > > > _______________________________________________ > Types-SIG mailing list > Types-SIG@python.org > http://www.python.org/mailman/listinfo/types-sig > From Edward Welbourne Mon Jan 17 00:15:23 2000 From: Edward Welbourne (Edward Welbourne) Date: Mon, 17 Jan 2000 00:15:23 +0000 (GMT) Subject: [Types-sig] Suggestions for python 2 Message-ID: I have been thinking about the namespace types. That's modules, packages, classes and instances. I believe they could profitably be united into *one* type. Along the way, I see some constructive uniformisations we could apply. I have written this up at http://www.chaos.org.uk/~eddy/dev/toy/python2.html and thence-linked locations. [The page also describes a fix to grail 0.5 which may be needed when reading the associated .py files.] My proposal contains enough slack to allow for the following things (among others) that some folk seem to want: * namespaces that you can't modify once they have been initialised * namespaces in which attribute-modification is type-checked * hybrids of those with assorted degrees of control * certain tools the functional programmers crave, notably a truly faithful implementation of currie() * some of the truly whacky `metaclass' proposals and, of course, the stuff we're used to (notably class, albeit demoted to a built-in) with minimal perturbation. The changes are, IMO, straightforward and pythonic: they reduce the number of special attributes the interpreter is obliged to know about and do *not* depend on strange hacks or anomalies. They do perturb syntax. I intend to push this proposal during IPC8. I would greatly appreciate constructive comments in advance, if only to temper my enthusiasm ;^} Eddy. -- Those who have seen versions of my pages prior to 2000/Jan/17 should note that I have simplified and clarified them somewhat of late. I believe I have eliminated all risk of exploding heads. YMMV. From guido@CNRI.Reston.VA.US Mon Jan 17 02:21:35 2000 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Sun, 16 Jan 2000 21:21:35 -0500 Subject: [Types-sig] Suggestions for python 2 In-Reply-To: Your message of "Mon, 17 Jan 2000 00:15:23 GMT." References: Message-ID: <200001170221.VAA10441@eric.cnri.reston.va.us> > I intend to push this proposal during IPC8. I would greatly appreciate > constructive comments in advance, if only to temper my enthusiasm ;^} Eddy, it's too late in Python's life for radical proposals. My head hasn't exploded yet, but it sure hurts whenever I try to read your proposal carefully. I'm not a functional programmer, and Python is not a functional programming language; and as far as I'm concerned it never will be one. Maybe you'll make the case better in person -- I'll be there to listen! (Practical comment: the syntax []: won't fly; you seem to want to be any expression, and an expression followed by a name is too fragile a construct to be comfortably parseable (even if it might be unambiguous).) --Guido van Rossum (home page: http://www.python.org/~guido/) From Edward Welbourne Mon Jan 17 11:59:05 2000 From: Edward Welbourne (Edward Welbourne) Date: Mon, 17 Jan 2000 11:59:05 GMT Subject: [PSA MEMBERS] Re: [Types-sig] Suggestions for python 2 In-Reply-To: <200001170221.VAA10441@eric.cnri.reston.va.us> References: <200001170221.VAA10441@eric.cnri.reston.va.us> Message-ID: <200001171159.LAA17397@lsls4p.lsl> > I'm not a functional programmer, and Python is not a functional > programming language; and as far as I'm concerned it never will be > one. and that's just fine with me. The reason for my list of `things this will empower' is: there are folk asking for stuff that would mangle python so that they can have those things. My belief is that we don't need to mangle python to achieve what I have in mind - and it'll incidentally give them those things, in so far as they insist on having them, without the rest of us having to notice. In particular, safe tunnels (and keyword-only arguments) are just *too* nice to not ask for. And far nicer than asking for built-in functional tools. > (Practical comment: the syntax []: > won't fly; you seem to want to be any expression, and an > expression followed by a name is too fragile a construct to be > comfortably parseable (even if it might be unambiguous).) No problem: always ask for more than need, so can back-pedal later. The generator could happily be a simple name (even one which has previously been the subject of an statement which says `I intend to use this as a generator' if desired). Likewise, bases-tuples could be as at present (rather than arbitrary expression yielding tuple, as asked for). > Maybe you'll make the case better in person I'm looking forward to trying ... ;^) See you next week, Eddy. From skaller@maxtal.com.au Mon Jan 17 14:36:40 2000 From: skaller@maxtal.com.au (skaller) Date: Tue, 18 Jan 2000 01:36:40 +1100 Subject: [Types-sig] Suggestions for python 2 References: Message-ID: <388328F8.85147F85@maxtal.com.au> Edward Welbourne wrote: > > I have been thinking about the namespace types. > That's modules, packages, classes and instances. > I believe they could profitably be united into *one* type. > Along the way, I see some constructive uniformisations we could apply. Sigh. See Viper. Modules, packages, classes, instance, raw objects, local scopes .. and some other scopes .. are all 'unified' in two ways: 1) Support for an 'environment' protocol supporting unqualified name lookup 2) Support for qualified lookup [x.y] In Viper, 'environment' is a fundamental abstraction of the execution environment: there's an actual class interface abstraction for it. Qualified lookup is supported by ad-hoc polymorphism, more or less encapsulated in the 'getattr' 'setattr' 'hasattr' functions. > * namespaces that you can't modify once they have been initialised > * namespaces in which attribute-modification is type-checked As you can see, in Viper, there are _two_ dual concepts of namespaces. In both cases, the fundamental implementation involves dictionaries, plus extra structure. > * hybrids of those with assorted degrees of control > * certain tools the functional programmers crave, notably a truly > faithful implementation of currie() not required: python lambdas subsume what can be done with currying. > * some of the truly whacky `metaclass' proposals There are plenty of them :-) See Viper. It already supports generic types, eg lists of X for any type X. This is done fairly easily by making ListOf a class with one attribute 'type' ListOf(X) an instance with type = X and that instance the type of a raw object with a normal list as it's only attribute. Methods of the ListOf class have two objects: the type object, (ListOf(X)), and the list instance (self). These are both bound by the two stage lookup, so that a method like def append(typeobj, self, value): if type(value) is not typeobj.type: raise TypeError else: self.list.append(value) > I intend to push this proposal during IPC8. I would greatly appreciate > constructive comments in advance, if only to temper my enthusiasm ;^} The lookup mechanism in Viper is very powerful. It is not necessarily 'right'. It might be useful to examine a working implementation? After all, it more or less models what C Python 1.5.2 does now, and just relaxes some restrictions on what a type object can be. [Viper raw objects can also be metamorphic -- that is, change type dynamically -- the type is 'just another attribute'] -- John (Max) Skaller, mailto:skaller@maxtal.com.au 10/1 Toxteth Rd Glebe NSW 2037 Australia voice: 61-2-9660-0850 download: ftp://ftp.cs.usyd.edu/au/jskaller From Edward Welbourne Mon Jan 17 18:37:20 2000 From: Edward Welbourne (Edward Welbourne) Date: Mon, 17 Jan 2000 18:37:20 GMT Subject: [PSA MEMBERS] Re: [Types-sig] Suggestions for python 2 In-Reply-To: <388328F8.85147F85@maxtal.com.au> References: <388328F8.85147F85@maxtal.com.au> Message-ID: <200001171837.SAA20866@lsls4p.lsl> >> * certain tools the functional programmers crave, notably a truly >> faithful implementation of currie() > > not required: python lambdas subsume what can be done with > currying. Unfortunately, unless I'm missing something, this is only true when you know the call-signature of the function you are packaging. Without safe tunnels, I can see no way of handling even the simple case of `provide the first argument to a function given no knowledge of how many arguments the caller is going to give it': the currie()d result has to use *args in its argument-list, yet we need arguments which supply it with the function to call and the given first argument; these have to appear before the *args (unless something has changed and no-one has told me), consequently they will be over-ridden by the first few arguments passed in by the caller of the constructed function: def currieone(func, one): def result(*args, f=func, o=one): return apply(f, (o,) + args) return result as safe tunnels would put it; if f and o precede *args (which they currently must, as I understand it), the result is doomed because lambda f=func, o=one, *args: apply(f, (o,) + args) called with args (a, b, c) would bind a in as f (over-riding the default supplied, func), b as o (ditto, one), leaving args = (c,), so it would attempt to call a(b, c) where func(one, a, b, c) was our intent. Can you show me a piece of valid python (using lambdas or otherwise) which achieves this effect ? Granted, it can all be done (one of the first toys I built in python) as a class ... but I need stuff like it before I can implement class ... and we get so much more out of liberalising the position of *args and **kwds relative to the name=value parameters. Like, for instance, the ability to have a function take arbitrarily many positional arguments along with some keyword-arguments without the former and the latter getting tangled up with one another - the things I call generators commonly use (*bases, dict=None, meth=None, **what) ... unless dict and meth keywords are used when the function is invoked, the function will see them as None, no matter how many arguments are passed. And, just for clarity, I don't particularly want this for the functional tools (cute though I find them, fun though I find it to build them, and useful though they are as sample things to check a system is powerful enough to support) so much as for the expressive clarity of, for instance, def join(*strings, glue=' '): return string.joinfields(strings, glue) invokable as join('hello', 'there', 'all', 'you', 'pythoneers') or, if different padding is wanted, join('in', "Tim's", 'style', glue='-'), without having to put [] around the sequence of strings that aren't the keyword-supplied glue. Eddy. From jeremy@cnri.reston.va.us Mon Jan 17 18:54:16 2000 From: jeremy@cnri.reston.va.us (Jeremy Hylton) Date: Mon, 17 Jan 2000 13:54:16 -0500 (EST) Subject: [PSA MEMBERS] Re: [Types-sig] Suggestions for python 2 In-Reply-To: <200001171837.SAA20866@lsls4p.lsl> References: <388328F8.85147F85@maxtal.com.au> <200001171837.SAA20866@lsls4p.lsl> Message-ID: <14467.25944.592000.993643@goon.cnri.reston.va.us> Can we prune the reply-to list? There's no need for this to be on psa-members, which is for PSA-related business. I suggest just the types-sig, although just python-list would be fine, too. Jeremy From jeremy@cnri.reston.va.us Mon Jan 17 20:38:35 2000 From: jeremy@cnri.reston.va.us (Jeremy Hylton) Date: Mon, 17 Jan 2000 15:38:35 -0500 (EST) Subject: [Types-sig] developers' day session on compiling Message-ID: <14467.32203.521062.696153@goon.cnri.reston.va.us> I am championing a Developers' Day session at the conference that should be relevant to the types-sig. In a nutshell, the goal is to develop compiler tools written in Python to make projects like adding type declarations easier to implement. There is a short Web page describing the session at http://www.python.org/workshops/2000-01/compiler.html. Any feedback or suggestions for discussion would be appreciated. Jeremy From skaller@maxtal.com.au Mon Jan 17 20:51:45 2000 From: skaller@maxtal.com.au (skaller) Date: Tue, 18 Jan 2000 07:51:45 +1100 Subject: [PSA MEMBERS] Re: [Types-sig] Suggestions for python 2 References: <388328F8.85147F85@maxtal.com.au> <200001171837.SAA20866@lsls4p.lsl> Message-ID: <388380E0.36AB1146@maxtal.com.au> Edward Welbourne wrote: > > >> * certain tools the functional programmers crave, notably a truly > >> faithful implementation of currie() > > > > not required: python lambdas subsume what can be done with > > currying. > > Unfortunately, unless I'm missing something, this is only true when you > know the call-signature of the function you are packaging. > Can you show me a piece of valid python (using lambdas or otherwise) > which achieves this effect ? It hadn't occured to me you wouldn't know the arguments. I always know the positional arguments (not necessarily the optional ones) -- there's no other way to call a function :-) -- John (Max) Skaller, mailto:skaller@maxtal.com.au 10/1 Toxteth Rd Glebe NSW 2037 Australia voice: 61-2-9660-0850 homepage: http://www.maxtal.com.au/~skaller download: ftp://ftp.cs.usyd.edu/au/jskaller From guido@CNRI.Reston.VA.US Wed Jan 19 17:22:12 2000 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Wed, 19 Jan 2000 12:22:12 -0500 Subject: [Types-sig] Static typing: Towards closure? Message-ID: <200001191722.MAA19286@eric.cnri.reston.va.us> I've had an off-line discussion with Greg Stein (mostly) and Paul Prescod about trying to converge on a single proposal. I printed their respective proposals and took them on a trip, and came back with (1) a set of responses, and (2) a unified proposal that takes parts of each. I've shared these with Paul and Greg, and gotten some of their responses. More recently, I also prepared (3) some powerpoint slides summarizing my unified proposal and highlighting some open issues, to be presented at some time during developers' day at the conference next week (possibly in Jeremy's session on parsers). I've been trying to find the time to clean things up more, and hoping that Greg and/or Paul would find the time, but I've come to the conclusion that it's better to release these materials now than sit on them any longer, waiting for perfection to appear. So, here are URLs for the three documents, plus a text version of the unified proposal. (1) My initial responses (partially retracted in (2)): http://www.python.org/~guido/Response.doc (MS Word) (2) My unified proposal, plus some unfinished mumblings at the end: http://www.python.org/~guido/Proposal.doc (MS Word) http://www.python.org/~guido/Proposal.txt (ASCII) (3) Powerpoint slides summarizing (2): http://www.python.org/~guido/static-typing/ (HTML + GIF images) http://www.python.org/~guido/static-typing/static-typing.ppt (PowerPoint) Here is the text version of (2); feel free to post responses to this. (Please trim your quoting!) ---------------------------------------------------------------------- A different take Let's look at typechecking again. Introduction There are two kinds of modules: checked and unchecked. The programmer indicates inside the module source code that a module is a checked module. Proposed syntax: place "decl" on a line by itself at the top of the module (after the doc string if any, but must occur before the first statement -- this includes import statements and other decl statements). A checked module is required to be internally typesafe, and typesafe with respect to other checked modules that it imports. Typesafety is a compile-time property; every attempt is made to ensure that run-time behavior of unchecked modules cannot violate typesafety of checked modules. This restricts the language somewhat. The Python implementation should guarantee that a checked module cannot raise TypeError and similar exceptions, except when explicitly requested through the use a dynamic cast operator (the spelling of which is not yet decided). Other exceptions may occur, however: IndexError, KeyError, ZeroDivisionError, OverflowError and perhaps others can occur on certain operations; MemoryError can occur almost anywhere (like SystemError and KeyboardInterrupt). The fate of NameError and AttributeError for declared, but uninitialized variables is undecided; these require flow control and thus don’t strictly fall under type checks, but a rough check for these is not particularly hard, and such a guarantee would be very useful. When a checked module imports an unchecked module, all objects in the unchecked module (and the unchecked module itself) are assumed to be of type 'any', and the checked module is typechecked accordingly. When an unchecked module imports a checked module, typechecks are inserted at access points (where the unchecked module calls or otherwise accesses the checked module) to ensure that the type constraints specified by the checked module are satisfied at runtime. These checks may be bypassed when the checked module is accessed from another checked module. Because even checked modules may be loaded dynamically, when a checked module imports another checked module, the newly imported module is tested against its expected signature. If its signature is incompatible with the signature expected by the importing module (e.g. because the imported module was modified and rechecked since the importing module was last checked) the import fails (with an ImportError exception). Example: take a checked module containing a simple GCD function: # module gcdlib.py decl def gcd(a: integer, b: integer) -> integer: while a: a, b = b%a, a return b This module passes the typesafety check. Now consider another checked module using that is it: # module test_gcdlib.py decl from gcdlib import gcd print gcd(12, 14) print gcd(20, 200) print gcd(6400, 2L**100) This module also passes the typesafety check (note that the last print statement is accepted because the type 'integer' matches both the int and long standard types). The following statement, when added to this module, will fail the typesafety test, and thus make the module fail the safety test: print gcd(3.5, 3) Assuming we have a typechecking Python interpreter, running or importing the file test_gcdlib.py thus modified will fail with a compile-type typecheck error before a single statement in it is executed. Now consider an unchecked module that is using the gcdlib.py module: # module foo.py from gcdlib import gcd print gcd(12, 14) print gcd(20, 200) print gcd(6400, 2L**100) print gcd(3.5, 3) This will print the first three answers: 2 20 256L and fail with a dynamic typecheck error on the fourth print statement. On the other hand, if we now remove the "declare typecheck" from the gcdlib.py file, the algorithm as stated will perform some result for the fourth call (generally ill-defined, because of rounding inaccuracies of floating point arithmetic), e.g.: 2 20 256L 0.5 If we now run the modified test_gcdlib.py code (with the invalid statement added) we see the same results: the module is checked, but it calls an unchecked module, which can yield any results it likes. Now consider the following module (still using the unchecked version of gcdlib.py): # module bar.py decl # this is a checked module from gcdlib import gcd decl i: integer i = gcd(2, 3) This module does not pass the typecheck, because gcdlib.py is not checked, so its gcd() function is assumed to return a value of type 'any'. Could we fix this by adding "decl gcd: def(int, int)->int" to this module? No, because there's no way at compile time to verify that the gcd function in fact matches this signature; generating a runtime error when the result doesn't have the right type isn't very helpful (given that we're asking to be typechecked). Some more examples of code that passes or fails the typecheck: def gcd(a: any, b: any) -> integer: while a: a, b = b%a, a return b This fails, because the type of b is 'any', which isn't a proper subset of the type 'integer'. def foo(a: int) -> int: return a/2.0 This fails, because the type of the return expression is float, not int. def foo(a: int) -> int: b = a return b This passes, even though the type of the local variable b is not declared -- basic type inference can deduce its type. def foo(a: int) -> int: L = [] L.append(a) return L[0] This *fails*, because the type of L is not declared, and the typechecking algorithm doesn't specify powerful enough type inference to be able to deduce that L's type is "list of int". Here's how to fix the example: def foo(a: int) -> int: decl L: [int] L = [] L.append(a) return L[0] Obviously, we have to define more precisely how much the assumed "basic type inference" can deduce. For now, we presume that it sees basic blocks and assignments to simple variables. Now let's look at a realistic example. This is find.py (taken from Lib/lib-old in the Python 1.5.2 distribution) modified to be checked and typesafe. We assume that the imported modules, both part of the standard library, are checked and hence typesafe. Bold text was added: decl import fnmatch import os _debug = 0 decl _prune: [str] _prune = ['(*)'] def find(pattern: string, dir: string = os.curdir) -> [string]: decl list: [string] list = [] decl names: [string] names = os.listdir(dir) names.sort() for name in names: if name in (os.curdir, os.pardir): continue fullname = os.path.join(dir, name) if fnmatch.fnmatch(name, pattern): list.append(fullname) if os.path.isdir(fullname) and not os.path.islink(fullname): for p in _prune: if fnmatch.fnmatch(name, p): if _debug: print "skip", `fullname` break else: if _debug: print "descend into", `fullname` list = list + find(pattern, fullname) return list Note that the types of local variables 'name' and 'fullname' are not declared; their types are deduced from the context: the type of 'name' is the item type of 'names', and the type of 'fullname' is the return type of os.path.join(). (By the way, this gives us another indication of the required power for basic type inference; it has to know the relation between the type of the sequence iterated ober by a for loop and the type of the loop control variable.) Ditto for 'p'. It should be noted that the type declaration for 'names' could be omitted without effect, since the return type of os.listdir() is known to be [string]. Could we omit the declaration for '_prune'? No; even though it is effectively a constant (by intention, it changes only when the module is edited), the typechecker isn't required to notice this. Or is it even constant? Earlier, we've discussed how the runtime can prevent changes to module globals that are apparently constants. Hmm, even if we disallow direct assignment to 'find._prune' from outside the module, that doesn't stop us from writing 'find._prune.append("spam")', so even if the typechecker can deduce that _prune is a list, it can't assume that it is a list of strings, unless it is declared so. Such are the joys of working with mutable data types. On the other hand, '_debug' doesn't need to be declared, even when we assume outside assignments are legal, because its only use is in a true/false test, which applies to any object. Hmm, this may change in the future; the new design for rich comparisons introduces types that cannot be used directly in a Boolean context, because the outcome of a comparison between two arrays will be allowed to return an array of Booleans; in order to prevent naive programmers to write "if A < B: ..." where A and B are arrays, this will raise an exception rather than always returning true. Anway, the "no outside assignments" rule would do away with this argument, and it is the most sane rule to be adopted. Syntax I like the syntax shown above; it is roughly what Greg proposes (mostly inspired by Tim's earlier proposal). There's an optional alternative which places all type annotations inside decl statements; this makes is easier to remove the decl statements for use with an unmodified Python interpreter. Thus, decl gcd: def(a: integer, b: integer) -> integer def gcd(a, b): while a: a, b = b%a, a return b is equivalent to the "in-line" version: def gcd(a: integer, b: integer) -> integer: while a: a, b = b%a, a return b I think that maybe we can allow this too, with roughly the same meaning: decl def gcd(a: integer, b: integer) -> integer I’d like to distinguish the two slightly: the form ``decl def name(...)’’ should mean that we declare a function; the form ``decl name: def(...)’’ should mean that we declare a variable that holds a function (callable). This is a useful distinction (even more so in classes). Either of the following declares the argument types without declaring the names of keyword arguments, so the function can only be called with positional arguments: decl gcd: def(integer, integer) -> integer decl def gcd(integer, integer) -> integer Note that if the decl statement doesn't give the keyword argument names, the presence of the argument names (even with default values) in the actual def statement doesn't change this. On the other hand, if the argument type declarations are included in the function definition, they keyword argument names are implied. Here's another way to declare the argument and return types. It is more verbose, and equivalent to the in-line form: def gcd(a, b): decl a: integer, b: integer decl return: integer while a: a, b = b%a, a return b I don't like Paul's 'as' keyword. See Greg's argument about this (it suggests possibly changing the value to conform to the type). I don't like Greg's '!' operator. Its semantics are defined in terms of runtime checks, but I want the semantics of typechecking to be done at compile-time, as explained above. This is not negotiable at the moment. An alternative form of syntax that doesn't require changing the interpreter at all places the decl statements inside string literals, e.g.: "decl gcd: def(integer, integer) -> integer" def gcd(a, b): while a: a, b = b%a, a return b Paul suggests something similar, but uses a tuple of two strings. I don’t see the point of that (besides, such a tuple ends up being evaluated and then thrown away at run time; a simple string is thrown away during code generation). There's one more idea that I want to discuss: once the in-line syntax is accepted, the 'decl' keyword may be redundant (in most cases anyway). We might just as well write "a: int" on a line by itself rather than "decl a: int". The Python parser won't have serious problems with this, as long as the thing on the left of the colon can be simplified to be an expression. (This is already done for assignment statements.) Minor syntactic nit: I like to use '|' to separate type alternatives, not 'or'. Classes Now let's look at the use of type declarations to annotate class definitions. A class will mostly want to declare two kinds of things: the signatures of its instance methods, and the types of its instance variables. I will briefly discuss the declaration of class methods and variables below. I propose the syntax used in the following example: class Stack: decl _store: [any] def __init__(self): self._store = [] def push(self, x: any): self._store.append(x) def pop(self) -> any: x = self._store[-1] del self._store[-1] return x Note that 'self' is still mentioned in the def statement, but its type is not declared; it is implied to be 'Stack'. It is possible to use decl statements for the methods instead of inline syntax; then the decl statement should *not* list 'self': class Stack: decl _store: [any] def __init__(self): self._store = [] decl def push(any) def push(self, x): self._store.append(x) decl def pop() -> any def pop(self): x = self._store[-1] del self._store[-1] return x Note that no decl statement or in-line syntax is used for __init__; this means that it takes no arguments (remember that __init__ never returns anything). A future extension of the type checking syntax can easily be used to declare private and protected variables, or static variables, or const variables: decl protected _stack: [any] decl public static class_stats: int decl const MAXDEPTH: int In checked modules, no dynamic games may be played with classes. (Eventually, we'll allow certain dynamic games; for now, it's best to disallow them completely so we can get on with the goal of typechecking.) The typechecker must ensure that all declared instance variables are properly initialized. For instance variables with a mutable types, this means that they must be assigned to at least once before being used in the __init__ method. For instance variables with an immutable type, if an assignment at the class level is present, this is allowed. A class in a checked module must declare all its instance variables. Instance methods are implicitly declared by the presence of 'def' statements. Here's another example: class Tree: decl readonly label: string decl private left, right, parent: Tree|None def __init__(self, lab: string, l: Tree|None = None, r: Tree|None = None): self.label = lab self.parent = None self.left = l self.right = r if l is not None: assert l.parent is None l.parent = self if r is not None: assert r.parent is None r.parent = self def unlink(self): self.parent = None def setleft(self, x: Tree): assert x.parent is None if self.left is not None: self.left.unlink() self.left = x x.parent = self def setright(self, x: Tree): assert x.parent is None if self. right is not None: self. right.unlink() self. right = x x.parent = self def prefixvisit(self, visitor: def(Tree)): visitor(self) if self.left is not None: self.left.prefixvisit(visitor) if self.right is not None: self.right.prefixvisit(visitor) Here we see a tricky issue cropping up. The links are declared to be either a Tree node or None. This means that whenever a link is dereferenced, a check must be made. The type inferencer thus must be smart enough to detect these checks and notice that in the branch, the tested variable has the more restricted type. Most languages introduce special syntax for this (e.g. Modula-3 uses the 'typecase' statement). Can we get away with things like "if x is not None:" or the more general "if isinstance(x, Tree)"? Subtyping If f is defined as "def f(x: any) -> any", and an argument is declared as "def(int)", is f an acceptable argument value? Yes. However, if the argument is declared as "def(int)->int", the answer is No! Note that no declared return type is different than a declared return type of None here; no declared return type means that the return type is not used. Otherwise I see the subtyping rules as pretty straightfoward. I do think that the subtyping rules will require that subclasses declare their overriding methods with compatible signatures as base classes. This may cause standard contravariance-related issues. Given: class B: def method(self, other: B) -> B: ... the following is valid: class D(B): def method(self, other: B) -> D: ... but class D can't declare that its method requires a D: class D(B): def method(self, other: D) -> D: ... (Read any text on contravariance if you don't understand this; this is a well-known surprising requirement that C++ and Java also have. Eiffel solves it with a runtime check; is this really better?) Idea: Eiffel allows covariance (e.g. declaring other as D in the derived class) and inserts a run-time check. We could do the same, as it is so useful. Most of the time this could probably be checked at compile time; basically all casts of a D instance to a B are suspicious, and all calls in the base class of such a method may be suspicious (unless the instance and the argument are 'self'). Parameterized types Of course, the Stack example is begging to be a parameterized type! Let's suggest a syntax. I don't like the syntax proposed before very much; what's wrong with C++ template brackets? class Stack: decl _store: [T] def __init__(self): self._store = [] def push(self, x: T): self._store.append(x) def pop(self) -> T: x = self._store[-1] del self._store[-1] return x A variant without in-line syntax is easy, for example: class Stack: decl ... or (if you prefer): decl class Stack class Stack: ... The problem with this is how to write the instantiation, *and* how to do the type checking when this is used from an unchecked module. Let's try: decl IntStack = Stack decl x: IntStack x = IntStack() x.push(1) print x.pop() x.push("spam") # ERROR decl s: string s = x.pop() # ERROR print isinstance(x, IntStack) # True print isinstance(x, Stack) # True y = Stack() # ERROR The first (slight) problem here is that the first decl statement here must introduce a new name in the *runtime* environment (which hitherto we have carefully avoided). This can be done; the syntax uses '=' to signal this to the parser. The second problem is that when a checked module creates an IntStack instance, and passes this out into an unchecked module, the instance must contain added typechecking code so that attempts to push non-ints are properly rejected (otherwise a later pop() in the checked code could yield a surprise). This means that either the statement decl InstStack = Stack must do template instantiation just like C++ (shrudder!); or the statement x = IntStack() must pass a hidden extra parameter giving the parameter type ('int') to the constructor, which store the type descriptor in a hidden instance variable, and all the methods must contain explicit type checking against the parameter type; this is slower but seems more Pythonic. In any case there will be little hope that we can fully support parameterized types in the experimental version of the syntax where decl statements are totally invisible to the parser. The statement "decl IntStack = Stack" must be replaced by something like "IntStack = Stack", at the very least. This makes the string literal experimental syntax hard to realize. Exceptions A checked module that passes the typecheck may still raise exceptions when used. Dereferencing None is a typecheck error, and so is using an unknown name, an uninitialized name (hopefully), or an unknown or uninitialized attribute; but indexing out of range, using an unknown key, dividing by zerio, and a variety of other conditions (e.g. MemoryError, IOError or KeyboardInterrupt) may cause exceptions. It would be nice if we could guarantee that no exceptions could be raised (with the exception of MemoryError or KeyboardInterrupt, which can never be prevented), but IndexError, KeyError and ZerodivisionError are hard to chek for at compile time. What do other languages do? Java and Modula-3 require such exceptions to be declared. (C++ too?) Maybe we should follow suit and do the same thing... (However, I believe that Java makes indexing errors non-declared exceptions, I believe, and ditto for null pointer dereferencing.) Open issues / ideas for the future Paul Prescod has some examples of parameterized functions, e.g. (in my syntax): def f (a: T) -> T: ... This is only useful if there is a way to instantiate such "template" functions; I could claim that if f(1) returns 1.0, it is valid because I choose "number" for T. I’m not sure that we need this much; parameterized classes seem to take care of most cases, so I suggest not to bother in version 1.0. Interfaces? Paul Prescod suggests using 'interface' as the keyword and otherwise using the class syntax, but without function bodies and initializations. Seems fine with me. To declare that we’re using an interface, we simply list it as a base class. Interfaces should be listed after all regular base classes. But I wonder if we need to bother in version 1.0. It may be useful to declare the type of a variable as conforming to several interfaces. This could be expressed with the '&' operator, e.g. ``decl a: I1 & I2''. Checked modules and classes form a good basis to revisit the idea of require/ensure (i.e., programmming by contract, as in Eiffel, and as propagated for Python by Paul Dubois). Can an unchecked module subclass a class defined by a checked module in such a way that it violates the type checks? This could be detected dynamically, either at class definition time or (at the latest) when a method is called or an instance variable is assigned to. Since int is a subtype of any, does this mean that Stack is a subtype of Stack? No, I don’t think so. Example: decl a: [int] decl b: [any] a = [1,2,3] b = a b.append("spam") The last statement is illegal because of the aliasing, but legal in the light of the type of b. The solution is either a dynamic typecheck or to say that [int] is not a subtype of [any]. Note that Java calls it a subtype and inserts the dynamic check -- apparently because it is lacking parameterized types. (See [Bruce96].) This probably has consequences for many generic or polymorphic functions; e.g. the bisect() function. So, perhaps the idea of parameterized functions does have a purpose? Jeremy suggests that at least for functions, parameterization should be explicit in the declaration (e.g. def bisect(a: [T], x: T) -> int) but implicit in the function application (e.g. decl a: [int]; a = [1,2,10]; i = bisect(a, 5)). There’s another problem here. Without extra typechecking, lists or dicts of specific types aren’t even safe as subtypes of 'any'; if I declare x: [int], then passing x into unchecked code is unsafe unless lists implement typechecking! However this is a bit different than the above example -- here we pass a checked object into unchecked territory, so the object is required to defend itself; in the previous example, the type error occurred within fully checked code (and [Bruce96] explains why). What syntax to use to declare exceptions? Is there any hope for compile time checking of out-of-bounds indexing? (The obsession of Pascal, in response to a hang-up from debugging too many Fortran programs :-) The requirement to insert dynamic type checks when an unchecked module calls a function defined in a checked module can easily be expensive; consider passing a list of 1000 ints to a function taking an argument of type [int]. Each of the ints must be checked! Jeremy suggested caching the union type of the elements in the list object, but this slows down everything since we don’t know whether the list will ever be used in a context where its type is needed (also deletions may cause the cached type to be to big, so that a full check may still occasionally be needed). It may be better to accept the overhead. (An alternative idea is to generate code in the checked function that does the type check only when items are extracted from the list, but of course this means that all checked code must typecheck all items extracted from all sequences, since sequences may be freely passed around between checked code.) Do we need a way to refer to the type of self? E.g. class C: def __add__(self, other: typeof(self)) -> typeof(self): ... This would be especially useful in interfaces... After reading part of [Bruce96]: maybe we need to separate the notion of subtyping and subclassing. Can the type of a class’ instance be determined automatically based on its class? Is there a way to detect the error pattern that Bertrand Meyer calls "polymorphic catcalls"? (A catcall is a call on self.foo(...args...) where a subclass could redefine foo() to be incompatible with the given arguments.) We should allow redefining methods with subtypes for the return value and supertypes for the arguments (but that’s not very useful). Example: overriding the __copy__() method to return the type of the current class. We could even add a mechanism to automatically vary the return type even if the method is not explicitly overridden. (But how to typecheck it?) After reading [Bruce96]: allowing MyType for argument and return value types is helpful and can be made safe, although may lead to subclasses not being subtypes. What *is* the type of self? is it the (statically known) current class? Is it something special? [Bruce96] explains that it is something special (MyType, a free variable). It is of course known to be a subtype of the defining class; but otherwise it needs to be considered a free variable. Not clear how this affects the type checking algorithm... References [Bruce96] Kim Bruce: Typing in object-oriented languages: Achieving expressiveness and safety. http://www.cs.williams.edu/~kim/README.html [Meyer] Bertrand Meyer: Beware of polymorphic catcalls. http://www.eiffel.com/doc/manuals/technology/typing/cat.html ---------------------------------------------------------------------- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@CNRI.Reston.VA.US Wed Jan 19 19:49:11 2000 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Wed, 19 Jan 2000 14:49:11 -0500 Subject: [Types-sig] Re: puzzle about subtyping and parameterization In-Reply-To: Your message of "Wed, 19 Jan 2000 14:25:42 EST." <14470.4022.551614.971388@goon.cnri.reston.va.us> References: <200001191729.MAA19324@eric.cnri.reston.va.us> <14470.2448.957015.160023@goon.cnri.reston.va.us> <200001191902.OAA19487@eric.cnri.reston.va.us> <14470.4022.551614.971388@goon.cnri.reston.va.us> Message-ID: <200001191949.OAA19604@eric.cnri.reston.va.us> [Jeremy] > I thought I had a solution until you mentioned strings :-(. For > numeric types, I was going to propose that number be used most of the > time, and that 1/2 == .5 and that 2**45 doesn't raise an > OverflowError. If most programs use this version of number, then we > can afford to give funny names to what you have called 'int' and > 'integer' because most programs won't use them. That would be acceptable for Python 2.0, but unfortunately not for 1.6; there are lots of places where 'int' is required by the runtime (e.g. indexing). > How about 'fixnum' for what is now IntType and 'int' for IntType and > LongType? I would do it the other way around: int == IntType, long == LongType, fixnum == int|long (or perhaps fixnum should encompass *all* integral types). > GvR> I'm reluctant to introduce case differences; many people don't > GvR> notice the difference between Int and int anyway. > > Yikes! I'm worried that you're only "reluctant" and not "morally > opposed." Funny, I edited it down from stronger language. I meant in addition to different words, e.g. concrete types (int, long etc.) are lowercase but abstract types start with an capital, e.g. Integer, Number, Any. (None will always be an exception. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From jeremy@cnri.reston.va.us Wed Jan 19 20:13:18 2000 From: jeremy@cnri.reston.va.us (Jeremy Hylton) Date: Wed, 19 Jan 2000 15:13:18 -0500 (EST) Subject: [Types-sig] Static typing: Towards closure? In-Reply-To: <200001191722.MAA19286@eric.cnri.reston.va.us> References: <200001191722.MAA19286@eric.cnri.reston.va.us> Message-ID: <14470.6878.734177.106589@goon.cnri.reston.va.us> >>>>> "GvR" == Guido van Rossum writes: GvR> (possibly in Jeremy's session on parsers). A nit, but worth noting: The session is on compiling not just parsing. I think these typing issues are worth discussing. Jeremy From skip@mojam.com (Skip Montanaro) Wed Jan 19 21:13:39 2000 From: skip@mojam.com (Skip Montanaro) (Skip Montanaro) Date: Wed, 19 Jan 2000 15:13:39 -0600 Subject: [Types-sig] Static typing: Towards closure? In-Reply-To: <200001191722.MAA19286@eric.cnri.reston.va.us> References: <200001191722.MAA19286@eric.cnri.reston.va.us> Message-ID: <200001192113.PAA31734@beluga.mojam.com> Guido, Having only looked quickly at the HTML versions of your slides, I'll make a couple quick comments: 1. On slide 7/16 you have different numbers of leading and trailing underscores for "builtin" in two places. Looks like one leading and two trailing... 2. Somewhere (and I'm not saying it necessarily belongs here), we need to ask what the tradeoffs are of fully typed (all modules) vs. partially typed (some modules) vs. untyped programs are in a number of areas. * complexity of the overall Python system - who maintains it, how much bigger do the various bits and pieces get, what about performance? * complexity of indivudal pieces of the proposal - what if some feature is discovered to be "too complex", will the whole thing fall apart if (for example) parameterized types can't be added with a reasonable amount of effort? * effort required to reach a desired level of type safety - is there a measurable increase in type safety in an n-module program adding checking to the m-th module when m << n or do you not see much benefit until m approaches m? when adding type checking to another module, which one should be next? * approachability for novice users - will novices still be able to use the standard library code as a learning tool or will it all start to look like C++? <0.1 wink> * "const"-like effect - will adding (some aspect of) typing to module A force me to add typing to module B, similar to the effect people often experience adding const keywords to existing C code? * third-party modules - if I want type safety but use an untyped third-party module in my program, what are my options? what are the implications? I'm a bit under the weather, so I'm in a more contrarian mood than usual in this regard. Apologies in advance... Skip Montanaro | http://www.mojam.com/ skip@mojam.com | http://www.musi-cal.com/ 847-971-7098 From mhammond@skippinet.com.au Wed Jan 19 21:25:41 2000 From: mhammond@skippinet.com.au (Mark Hammond) Date: Wed, 19 Jan 2000 13:25:41 -0800 Subject: [Types-sig] Static typing: Towards closure? In-Reply-To: <200001191722.MAA19286@eric.cnri.reston.va.us> Message-ID: I like this proposal - it clears up some of the issues that I didn't really follow in the other proposals. I didn't follow them because of a lack of time, but this document states them explicitly for me. Combining assignments and decls: -------------------------------- I note that default args with typing look like: def find( ..., dir: string = os.curdir) -> [string]: However, simple typed assignments takes 2 statements: decl list: [string] list = [] would it be possible to combine them thus: decl list : [string] = [] decl i : int = 0 I dont want to _require_ the assignment in the decl, but it seems a worthwhile shortcut, and my complete lack of knowledge about parsers tells me it should be doable :-) Typo?: ------ I note that the first decl in the sample find.py has: decl _prune: [str] Is this a typo? Do you mean "[string]" here? I just want to be clear that there arent different semantics intended. Style Preference: ----------------- Minor "style" issue: I would prefer "int" over "integer" - Python is a nice succinct language, and "int" seems just as descriptive as "integer" - so unless the issue is a collision with the builtin int() function, is there a reason to prefer the longer name over the short one? Exceptions: ----------- You say: """It would be nice if we could guarantee that no exceptions could be raised (with the exception of MemoryError or KeyboardInterrupt, which can never be prevented), but IndexError, KeyError and ZerodivisionError are hard to chek for at compile time. What do other languages do? """ IMO we should punt this. Attempting to handle these exceptions is too hard, and too unpythonic. Open Issues - Interfaces: ------------------------- I would like to see interfaces included in version 1. I have some (unfortunately still private) reasons for wanting this, but also see it as very useful capability. I can't see a huge cost in supporting interfaces. However, I dont understand: """ It may be useful to declare the type of a variable as conforming to several interfaces. This could be expressed with the '&' operator, e.g. ``decl a: I1 & I2''. """ I dont see how a variable can support several interfaces. How is this different than supporting multiple types, which is reflected via the '|' operator? A _class_ can support multiple interfaces, but a variable appears to be closer to the "|" relationship. thus: class Foo(I1): ... class Bar(I2): ... class Spam: ... decl a: I1 | I2 ... a = Foo() # Legal a = Bar() # Legal a = Spam() # NOT legal # I can't picture an assignment that forces the "&" operator. Unless the intent is: class Foo(I1): ... class Bar(I2): ... class Spam(I1,I2): ... decl a: I1 & I2 ... a = Foo() # NOT Legal - only I1 a = Bar() # NOT Legal - only I2 a = Spam() # Legal - _both_ interfaces. I dont see a good reason for this. Can you flesh out your intentions here some more? Anecdote: --------- """ Checked modules and classes form a good basis to revisit the idea of require/ensure (i.e., programmming by contract, as in Eiffel, and as propagated for Python by Paul Dubois). """ Ive been doing some work with Bertrand Meyer, and he mentioned he knows Paul. His comment was, paraphrased: "Paul is very smart and I have lots of respect for him - except that he uses Python" :-) Maybe this is just Paul's way of overcoming that last objection? :-) [And I can tell you - giving a demo to Professors Meyers and Mingins of some of my work was definately a nerve-wracking experience :-)] Excellent stuff - Im very pleased to see this moving along, and I think Greg, Paul and others have done an excellent job in thrashing out the issues to the point where Guido can write a paper like this. Well done types-sig! Mark. From guido@CNRI.Reston.VA.US Wed Jan 19 21:32:17 2000 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Wed, 19 Jan 2000 16:32:17 -0500 Subject: [Types-sig] Static typing: Towards closure? In-Reply-To: Your message of "Wed, 19 Jan 2000 15:13:39 CST." <200001192113.PAA31734@beluga.mojam.com> References: <200001191722.MAA19286@eric.cnri.reston.va.us> <200001192113.PAA31734@beluga.mojam.com> Message-ID: <200001192132.QAA20055@eric.cnri.reston.va.us> > From: Skip Montanaro > Having only looked quickly at the HTML versions of your slides, I'll make a > couple quick comments: > > 2. Somewhere (and I'm not saying it necessarily belongs here), we need to > ask what the tradeoffs are of fully typed (all modules) vs. partially > typed (some modules) vs. untyped programs are in a number of areas. > > * complexity of the overall Python system - who maintains it, how much > bigger do the various bits and pieces get, what about performance? Good question. Open for debate. (Who? Me? :-) > * complexity of indivudal pieces of the proposal - what if some feature > is discovered to be "too complex", will the whole thing fall apart if > (for example) parameterized types can't be added with a reasonable > amount of effort? Without parameterized types, it's very weak. But I'm confident that we can get this to work. > * effort required to reach a desired level of type safety - is there a > measurable increase in type safety in an n-module program adding > checking to the m-th module when m << n or do you not see much benefit > until m approaches m? when adding type checking to another module, > which one should be next? I think a lot of help will come from having the entire standard library typechecked. I think we may be able to statically check all (or at least most) uses of checked modules, even in unchecked modules. This should discover lots of things, like calling open() with the wrong arguments, etc. A type inferencing system for unchecked modules could also use the types declared for checked module to its advantage. > * approachability for novice users - will novices still be able to use > the standard library code as a learning tool or will it all start to > look like C++? <0.1 wink> I find I have to add less than 10% new code (all decl statements) in the few examples I've tried (e.g. fnmatch). Not too bad. > * "const"-like effect - will adding (some aspect of) typing to module A > force me to add typing to module B, similar to the effect people often > experience adding const keywords to existing C code? I don't expect this -- unless the typing added to A is more restricting than the actual use warrants. I would hope that most static errors come in the place of real dynamic errors. But this is something to gain more experience in. > * third-party modules - if I want type safety but use an untyped > third-party module in my program, what are my options? what are the > implications? You'll get run-time exceptions when you use the third-party module wrong, just like now. > I'm a bit under the weather, so I'm in a more contrarian mood than usual in > this regard. Apologies in advance... No apologies needed; that was a useful checklist. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@CNRI.Reston.VA.US Wed Jan 19 21:50:24 2000 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Wed, 19 Jan 2000 16:50:24 -0500 Subject: [Types-sig] Static typing: Towards closure? In-Reply-To: Your message of "Wed, 19 Jan 2000 13:25:41 PST." References: Message-ID: <200001192150.QAA20089@eric.cnri.reston.va.us> > Combining assignments and decls: > -------------------------------- > I note that default args with typing look like: > > def find( ..., dir: string = os.curdir) -> [string]: > > However, simple typed assignments takes 2 statements: > decl list: [string] > list = [] > > would it be possible to combine them thus: > decl list : [string] = [] > decl i : int = 0 > > I dont want to _require_ the assignment in the decl, but it seems a > worthwhile shortcut, and my complete lack of knowledge about parsers tells > me it should be doable :-) Once decl is fully integrated, no problem. Before that, it has the same problem as the inline syntax: if you simply strip out all the decl statements, you should be left with a correct "classic" Python module, and if the code with decl statements typechecks correctly, the classic Python version should be type-error free. > Typo?: > ------ > I note that the first decl in the sample find.py has: > > decl _prune: [str] > > Is this a typo? Do you mean "[string]" here? I just want to be clear that > there arent different semantics intended. Not a typo; I like my concrete type names to have the same name as corresponding builtin functions. The long form "string" will stand for "either 8-bit or Unicode string". But this is debatable (see Jeremy's post). > Style Preference: > ----------------- > Minor "style" issue: I would prefer "int" over "integer" - Python is a nice > succinct language, and "int" seems just as descriptive as "integer" - so > unless the issue is a collision with the builtin int() function, is there a > reason to prefer the longer name over the short one? Similar issue -- I've been using "int" for the concrete type, and "integer" for the union of all int-like types: int | long | ... But this is debatable (see Jeremy's post). > Exceptions: > ----------- > You say: > """It would be nice if we could guarantee that no exceptions could be raised > (with the exception of MemoryError or KeyboardInterrupt, which can never be > prevented), but IndexError, KeyError and ZerodivisionError are hard to chek > for at compile time. What do other languages do? > """ > IMO we should punt this. Attempting to handle these exceptions is too hard, > and too unpythonic. Agreed. Avoiding IndexError or KeyError is equivalent to solving the halting problem. On the other hand, NameError and AttributeError are a gray area -- the proposal doesn't deal with them but it would be very nice if we could check these off at compile time, too. > Open Issues - Interfaces: > ------------------------- > I would like to see interfaces included in version 1. I have some > (unfortunately still private) reasons for wanting this, but also see it as > very useful capability. I can't see a huge cost in supporting interfaces. I agree. > However, I dont understand: > """ > It may be useful to declare the type of a variable as conforming to several > interfaces. This could be expressed with the '&' operator, e.g. ``decl a: > I1 & I2''. > """ > I dont see how a variable can support several interfaces. How is this > different than supporting multiple types, which is reflected via the '|' > operator? A _class_ can support multiple interfaces, but a variable appears > to be closer to the "|" relationship. Not really. Java supports one class implementing multiple interfaces, and it's very useful. Let's say I define separate interfaces for readable files and writable files: interface readf: def read(n: int = -1) -> str def readline() -> str def readlines(n: int = -1) -> str interface writef: def write(s: str) def writelines(l: [str]) Now if I want to say that I am expecting a file that's both readable and writable, I can write def myfunc(f: readf & writef): ... The other form (readf | writef) would request either a readable or a writable file! > thus: > class Foo(I1): > ... > class Bar(I2): > ... > class Spam: > ... > decl a: I1 | I2 > ... > a = Foo() # Legal > a = Bar() # Legal > a = Spam() # NOT legal > > # I can't picture an assignment that forces the "&" operator. Unless the > intent is: > > class Foo(I1): > ... > class Bar(I2): > ... > class Spam(I1,I2): > ... > decl a: I1 & I2 > ... > a = Foo() # NOT Legal - only I1 > a = Bar() # NOT Legal - only I2 > a = Spam() # Legal - _both_ interfaces. > > I dont see a good reason for this. Can you flesh out your intentions here > some more? Yes, this is what I wanted. See above for example. > Anecdote: > --------- > """ > Checked modules and classes form a good basis to revisit the idea of > require/ensure (i.e., programmming by contract, as in Eiffel, and as > propagated for Python by Paul Dubois). > """ > Ive been doing some work with Bertrand Meyer, and he mentioned he knows > Paul. His comment was, paraphrased: "Paul is very smart and I have lots of > respect for him - except that he uses Python" :-) Maybe this is just Paul's > way of overcoming that last objection? :-) > > [And I can tell you - giving a demo to Professors Meyers and Mingins of some > of my work was definately a nerve-wracking experience :-)] Wow! When did that happen? Meyers once invited me to teach Python at one of his TOOLS conferences, but somehow I had to leave before the conference was over, and he didn't show up before I had left. So I've never met him. > Excellent stuff - Im very pleased to see this moving along, and I think > Greg, Paul and others have done an excellent job in thrashing out the issues > to the point where Guido can write a paper like this. Well done types-sig! My sentiments exactly! --Guido van Rossum (home page: http://www.python.org/~guido/) From scott@chronis.pobox.com Wed Jan 19 22:18:32 2000 From: scott@chronis.pobox.com (scott) Date: Wed, 19 Jan 2000 17:18:32 -0500 Subject: [scott@chronis.pobox.com: Re: [Types-sig] Static typing: Towards closure?] Message-ID: <20000119171832.B63748@chronis.pobox.com> Resending a portion of this message to Guido to open up the issues a bit: scott ----- Forwarded message from scott ----- Date: Wed, 19 Jan 2000 15:17:28 -0500 From: scott To: Guido van Rossum Cc: josh@icgroup.com Subject: Re: [Types-sig] Static typing: Towards closure? X-Mailer: Mutt 0.95.7i In-Reply-To: <200001191722.MAA19286@eric.cnri.reston.va.us> On Wed, Jan 19, 2000 at 12:22:12PM -0500, Guido van Rossum wrote: [...] ------- Also, I have some general comments and questions about your proposal: *optional* In reading your proposal, you make multiple references to an "unmodified interpreter" and to having mutltiple grammars. I understand that this is an intuitive response to making static type checking optional in python, but I think it is not feesible given your 'typecase' strawman slide. I have been struggling with this 'typecase' issue with many different approaches and have come to the conclusion that if a type checker is going to allow OR'd types, and that type checker is going to distinguish confusion between those OR'd types on the part of a programmer, then something like a typecase operator is fundamentally necessary. Furthermore, if a program utilizes something like a typecase operator, then the program must have access to the declarations at run time, even if the type checking is optional. For me, this redefines 'optional' to mean 'you can use these constructs if you want', not 'you can use this interpreter and your type errors will be reported, otherwise you can use an unmodified interpreter on the same program'. This new definition of 'optional' seems fitting to the word and acceptable to me, but more subtle than offering the option of using a different interpreter. Do you agree? *paramterization* your section on parameterization uses the example of classes, when use of a type paramater is potentially widely available to functions and built in container types like lists and tuples. I believe a more general form of 'paramterization' which works along the lines of ML's "'a", "'b", etc is quite feasible for a static type checking system, as it would involve just a few additional checks and operations in the most atomic type checking functions. Please take into account in the design for paramterization that it should be available for more basic types than classes. ML has remarkably clean treatment of the idea, btw. *Constructors* your type proposal seems to mysteriously be missing the notion of type constructors, that is, something like this: datatype FOO = Int | None datatype BAR = Int | None decl a: FOO decl b: BAR a = 1 b = a # ERROR! b = BAR(a) # ok, explicit cast or 'constructor' It is possible to parse the cast at compile time, since it requires no knowledge of the value of `a', only it's type. The above example is slightly contrary to a grammar I put together, and I don't believe it's the right way to denote constructors, but it does demonstrate the intended semantics. using constructors can make dealing with logically OR'd types much easier and cleaner, and allows the user to define more specific types than allowed with builtin types. I think it's quite a worthwhile pursuit. [...] ----- End forwarded message ----- From guido@CNRI.Reston.VA.US Wed Jan 19 22:49:53 2000 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Wed, 19 Jan 2000 17:49:53 -0500 Subject: [Types-sig] Static typing: Towards closure? In-Reply-To: Your message of "Wed, 19 Jan 2000 17:18:32 EST." <20000119171832.B63748@chronis.pobox.com> References: <20000119171832.B63748@chronis.pobox.com> Message-ID: <200001192249.RAA20508@eric.cnri.reston.va.us> > From: scott > ------- > Also, I have some general comments and questions about your proposal: > > *optional* > In reading your proposal, you make multiple references to an > "unmodified interpreter" and to having mutltiple grammars. I > understand that this is an intuitive response to making static type > checking optional in python, but I think it is not feesible given > your 'typecase' strawman slide. > > I have been struggling with this 'typecase' issue with many different > approaches and have come to the conclusion that if a type checker is > going to allow OR'd types, and that type checker is going to > distinguish confusion between those OR'd types on the part of a > programmer, then something like a typecase operator is fundamentally > necessary. Furthermore, if a program utilizes something like a > typecase operator, then the program must have access to the > declarations at run time, even if the type checking is optional. For > me, this redefines 'optional' to mean 'you can use these constructs if > you want', not 'you can use this interpreter and your type errors will > be reported, otherwise you can use an unmodified interpreter on the > same program'. > > This new definition of 'optional' seems fitting to the word and > acceptable to me, but more subtle than offering the option of using a > different interpreter. Do you agree? I couldn't agree more. The whole idea of "optional static typing" always meant that when you were ready for it, you could simply sprinkle some decl statements in your source code and the interpreter would start static typechecking for you. It would be a standard part of the standard language implementation. (Probably in Python 2.0, aka Python 3000. (Version numbers with 2000 in then are soo last millennium... :-)) Of course, while we're experimenting with all this, the new syntax may require early adopters to use a special implementation. Maybe we're naive in believing that it would be as simple as deleting all lines that begin with decl... But for a reasonable subset of the proposal (obviously not including typecase or the ! operator) that's possible. > *paramterization* > your section on parameterization uses the example of classes, when > use of a type paramater is potentially widely available to functions > and built in container types like lists and tuples. I believe a more > general form of 'paramterization' which works along the lines of ML's > "'a", "'b", etc is quite feasible for a static type checking system, > as it would involve just a few additional checks and operations in the > most atomic type checking functions. Please take into account in the > design for paramterization that it should be available for more basic > types than classes. ML has remarkably clean treatment of the idea, > btw. I'm not familiar with ML's 'a, 'b and I don't quite understand what you are saying here. In the long form of the proposal there's some text about parameterized functions, e.g.: def foo(a: T, b: T) -> T: ... The builtin types are all supposed to be derived (somehow!) from parameterized interfaces, e.g. interface sequence: __getitem__(int) -> T __setitem__(int, T) append(T) expand(sequence) # or perhaps: expand(SELFTYPE) # etc. > *Constructors* > your type proposal seems to mysteriously be missing the notion of > type constructors, that is, something like this: > > datatype FOO = Int | None > datatype BAR = Int | None > > decl a: FOO > decl b: BAR > > a = 1 > b = a # ERROR! Actually, in my mind, this would be allowed. I believe in structural type matching, not matching by name. (That's what I've been believing, anyway.) Possibly when you introduce classes there's no way to have another class match even if it has the same structure, but for simple unions I don't see the need. (Or is your "datatype" a special declaration different from my typedef, which I spell decl FOO = int | None ?) > b = BAR(a) # ok, explicit cast or 'constructor' > > It is possible to parse the cast at compile time, since it requires > no knowledge of the value of `a', only it's type. > > The above example is slightly contrary to a grammar I put together, > and I don't believe it's the right way to denote constructors, but it > does demonstrate the intended semantics. > > using constructors can make dealing with logically OR'd types much > easier and cleaner, and allows the user to define more specific types > than allowed with builtin types. I think it's quite a worthwhile pursuit. I have chosen names for the std types that are the same as existing built-in functions, because I want to get a similar effect. E.g. x = int(y) is currently a call of the built-in function int() which happens to return its argument converted to an int, or die with an exception; in the new system, it would have the same effect but it would be a constructor for the int datatype (and the built-in object int would be the same object as types.IntType). It's true that these constructors have something of a dynamic cast in them (they can fail if the argument is unacceptable). I think that in my proposal, x = int(a) would have about the same semantics as Greg's x = a ! int --Guido van Rossum (home page: http://www.python.org/~guido/) From scott@chronis.pobox.com Wed Jan 19 23:49:01 2000 From: scott@chronis.pobox.com (scott) Date: Wed, 19 Jan 2000 18:49:01 -0500 Subject: [Types-sig] [scott@chronis.pobox.com: Re: optional] Message-ID: <20000119184901.A64311@chronis.pobox.com> Intended for the list. scott ----- Forwarded message from scott ----- Date: Wed, 19 Jan 2000 18:47:49 -0500 From: scott To: Guido van Rossum Subject: Re: optional X-Mailer: Mutt 0.95.7i In-Reply-To: <200001192249.RAA20508@eric.cnri.reston.va.us> These messages get really long, so I'm chopping them up: On Wed, Jan 19, 2000 at 05:49:53PM -0500, Guido van Rossum wrote: > > From: scott > > > ------- > > Also, I have some general comments and questions about your proposal: > > > > *optional* > > In reading your proposal, you make multiple references to an > > "unmodified interpreter" and to having mutltiple grammars. I > > understand that this is an intuitive response to making static type > > checking optional in python, but I think it is not feesible given > > your 'typecase' strawman slide. > > > > I have been struggling with this 'typecase' issue with many different > > approaches and have come to the conclusion that if a type checker is > > going to allow OR'd types, and that type checker is going to > > distinguish confusion between those OR'd types on the part of a > > programmer, then something like a typecase operator is fundamentally > > necessary. Furthermore, if a program utilizes something like a > > typecase operator, then the program must have access to the > > declarations at run time, even if the type checking is optional. For > > me, this redefines 'optional' to mean 'you can use these constructs if > > you want', not 'you can use this interpreter and your type errors will > > be reported, otherwise you can use an unmodified interpreter on the > > same program'. > > > > This new definition of 'optional' seems fitting to the word and > > acceptable to me, but more subtle than offering the option of using a > > different interpreter. Do you agree? > > I couldn't agree more. The whole idea of "optional static typing" > always meant that when you were ready for it, you could simply > sprinkle some decl statements in your source code and the interpreter > would start static typechecking for you. It would be a standard part > of the standard language implementation. (Probably in Python 2.0, aka > Python 3000. (Version numbers with 2000 in then are soo last > millennium... :-)) > > Of course, while we're experimenting with all this, the new syntax may > require early adopters to use a special implementation. Maybe we're > naive in believing that it would be as simple as deleting all lines > that begin with decl... But for a reasonable subset of the proposal > (obviously not including typecase or the ! operator) that's possible. This is good news. without something akin to that 'typecase' idea, there is no type checking of OR'd types, and without that... no recursive types, no typechecking dict.get, etc. scott ----- End forwarded message ----- From scott@chronis.pobox.com Wed Jan 19 23:51:11 2000 From: scott@chronis.pobox.com (scott) Date: Wed, 19 Jan 2000 18:51:11 -0500 Subject: [Types-sig] Static typing: Towards closure? - parameterization In-Reply-To: <200001192249.RAA20508@eric.cnri.reston.va.us> References: <20000119171832.B63748@chronis.pobox.com> <200001192249.RAA20508@eric.cnri.reston.va.us> Message-ID: <20000119185111.B64311@chronis.pobox.com> On Wed, Jan 19, 2000 at 05:49:53PM -0500, Guido van Rossum wrote: > > From: scott > > > ------- > > Also, I have some general comments and questions about your proposal: > > *paramterization* > > your section on parameterization uses the example of classes, when > > use of a type paramater is potentially widely available to functions > > and built in container types like lists and tuples. I believe a more > > general form of 'paramterization' which works along the lines of ML's > > "'a", "'b", etc is quite feasible for a static type checking system, > > as it would involve just a few additional checks and operations in the > > most atomic type checking functions. Please take into account in the > > design for paramterization that it should be available for more basic > > types than classes. ML has remarkably clean treatment of the idea, > > btw. > > I'm not familiar with ML's 'a, 'b and I don't quite understand what > you are saying here. In the long form of the proposal there's some > text about parameterized functions, e.g.: > While ML has some qualities python has no business pursueing (like how it deals with namespaces, currying functions, etc), there is a reason it's commonly used to teach about typed programming, the semantics of it's type checking system are powerful, concise, and accurate when compared to most other languages. I definitely believe it's worth a look for the semantics of it's type checking, though not for anything else :) > def foo(a: T, b: T) -> T: ... > > The builtin types are all supposed to be derived (somehow!) from > parameterized interfaces, e.g. > > interface sequence: > __getitem__(int) -> T > __setitem__(int, T) > append(T) > expand(sequence) > # or perhaps: > expand(SELFTYPE) > # etc. > I'm not sure we're on the same page here: When I think of this parameterization, I expect that: decl mylist: sequence mylist = [0] mylist.append("foo") will fail a type check, because `T' means a that whatever type is used in the interface must remain the same across the usage of that declared variable. that's not how the builtin objects work. Am I missing something? It would not be too hard to implement the following semantics: decl my_uniform_list: [ ~T ] my_uniform_list = [1,2,3] my_uniform_list.append("foo") # ERROR scott From guido@CNRI.Reston.VA.US Wed Jan 19 23:56:40 2000 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Wed, 19 Jan 2000 18:56:40 -0500 Subject: [Types-sig] Static typing: Towards closure? - parameterization In-Reply-To: Your message of "Wed, 19 Jan 2000 18:51:11 EST." <20000119185111.B64311@chronis.pobox.com> References: <20000119171832.B63748@chronis.pobox.com> <200001192249.RAA20508@eric.cnri.reston.va.us> <20000119185111.B64311@chronis.pobox.com> Message-ID: <200001192356.SAA20666@eric.cnri.reston.va.us> > While ML has some qualities python has no business pursueing (like how > it deals with namespaces, currying functions, etc), there is a reason > it's commonly used to teach about typed programming, the semantics of > it's type checking system are powerful, concise, and accurate when > compared to most other languages. I definitely believe it's worth a > look for the semantics of it's type checking, though not for anything > else :) I have Ullmann's "Elements of ML Programming". Which chapter should I read? > > def foo(a: T, b: T) -> T: ... > > > > The builtin types are all supposed to be derived (somehow!) from > > parameterized interfaces, e.g. > > > > interface sequence: > > __getitem__(int) -> T > > __setitem__(int, T) > > append(T) > > expand(sequence) > > # or perhaps: > > expand(SELFTYPE) > > # etc. > > > I'm not sure we're on the same page here: When I think of this > parameterization, I expect that: > > decl mylist: sequence > mylist = [0] > mylist.append("foo") > > will fail a type check, because `T' means a that whatever type is used > in the interface must remain the same across the usage of that > declared variable. that's not how the builtin objects work. Am I > missing something? Your ``decl mylist: sequence'' doesn't specify the type parameter; this is either an error or defaults to any. Look at this example instead: decl mylist: sequence mylist = [0] mylist.append("foo") # static error Granted, the [0] should be specially marked as a list of ints; but that's not too hard (one more field in the list header). > It would not be too hard to implement the following semantics: > > decl my_uniform_list: [ ~T ] > my_uniform_list = [1,2,3] > my_uniform_list.append("foo") # ERROR I guess that's what I just said. But what is ~T? --Guido van Rossum (home page: http://www.python.org/~guido/) From scott@chronis.pobox.com Thu Jan 20 00:11:14 2000 From: scott@chronis.pobox.com (scott) Date: Wed, 19 Jan 2000 19:11:14 -0500 Subject: [Types-sig] Static typing: Towards closure? - parameterization In-Reply-To: <200001192356.SAA20666@eric.cnri.reston.va.us> References: <20000119171832.B63748@chronis.pobox.com> <200001192249.RAA20508@eric.cnri.reston.va.us> <20000119185111.B64311@chronis.pobox.com> <200001192356.SAA20666@eric.cnri.reston.va.us> Message-ID: <20000119191114.D64311@chronis.pobox.com> On Wed, Jan 19, 2000 at 06:56:40PM -0500, Guido van Rossum wrote: > > While ML has some qualities python has no business pursueing (like how > > it deals with namespaces, currying functions, etc), there is a reason > > it's commonly used to teach about typed programming, the semantics of > > it's type checking system are powerful, concise, and accurate when > > compared to most other languages. I definitely believe it's worth a > > look for the semantics of it's type checking, though not for anything > > else :) > > I have Ullmann's "Elements of ML Programming". Which chapter should I > read? I have a different text, but the chapters that seemed most potentially relevant to python's type checking were on lists and arrays, user defined types, recursive datatypes, and it's usage of OR'd types in general. > > > > def foo(a: T, b: T) -> T: ... > > > > > > The builtin types are all supposed to be derived (somehow!) from > > > parameterized interfaces, e.g. > > > > > > interface sequence: > > > __getitem__(int) -> T > > > __setitem__(int, T) > > > append(T) > > > expand(sequence) > > > # or perhaps: > > > expand(SELFTYPE) > > > # etc. > > > > > I'm not sure we're on the same page here: When I think of this > > parameterization, I expect that: > > > > decl mylist: sequence > > mylist = [0] > > mylist.append("foo") > > > > will fail a type check, because `T' means a that whatever type is used > > in the interface must remain the same across the usage of that > > declared variable. that's not how the builtin objects work. Am I > > missing something? > > Your ``decl mylist: sequence'' doesn't specify the type parameter; > this is either an error or defaults to any. > > Look at this example instead: > > decl mylist: sequence > mylist = [0] > mylist.append("foo") # static error > > Granted, the [0] should be specially marked as a list of ints; but > that's not too hard (one more field in the list header). > > > It would not be too hard to implement the following semantics: > > > > decl my_uniform_list: [ ~T ] > > my_uniform_list = [1,2,3] > > my_uniform_list.append("foo") # ERROR > > I guess that's what I just said. But what is ~T? ~T is a way of saying `T' is a paramater without the <> and without the need to fill it in with a particular type upon declaration. scott > > --Guido van Rossum (home page: http://www.python.org/~guido/) > > _______________________________________________ > Types-SIG mailing list > Types-SIG@python.org > http://www.python.org/mailman/listinfo/types-sig From scott@chronis.pobox.com Thu Jan 20 00:37:36 2000 From: scott@chronis.pobox.com (scott) Date: Wed, 19 Jan 2000 19:37:36 -0500 Subject: [Types-sig] question regarding adding words to the grammar Message-ID: <20000119193736.E64311@chronis.pobox.com> While I have worked with the pythons grammar to play with type checking, I found that no matter where a word is introduced in the grammar, it causes that word to become a keyword, even if that word cannot appear where a regular variable name might. For example, one thing that I once attempted as a way of declaring classes relationships to interfaces was: classdef: class NAME ['(' NAME (',' NAME)* ')'] ['implements' NAME] ':' suite However, when I compiled python with this grammar, it wouldn't let me use the word 'implements' as a variable name (gives SyntaxError). So the question is: If you can introduce context-sensitive words, how do you do it without making a mess out of the grammar and hacking compile.c? scott From scott@chronis.pobox.com Thu Jan 20 02:20:04 2000 From: scott@chronis.pobox.com (scott) Date: Wed, 19 Jan 2000 21:20:04 -0500 Subject: [Types-sig] Static typing: Towards closure? In-Reply-To: <200001191722.MAA19286@eric.cnri.reston.va.us> References: <200001191722.MAA19286@eric.cnri.reston.va.us> Message-ID: <20000119212004.A66477@chronis.pobox.com> On Wed, Jan 19, 2000 at 12:22:12PM -0500, Guido van Rossum wrote: > Here we see a tricky issue cropping up. The links are declared to be > either a Tree node or None. This means that whenever a link is > dereferenced, a check must be made. The type inferencer thus must be > smart enough to detect these checks and notice that in the branch, the > tested variable has the more restricted type. Most languages introduce > special syntax for this (e.g. Modula-3 uses the 'typecase' statement). > Can we get away with things like "if x is not None:" or the more > general "if isinstance(x, Tree)"? I've actually spent a good deal of time pondering this issue and coding segments that might potentially deal with having a type checker understand idioms such as 'if x is None'. These idioms are tricky for a number of reasons, that basically relate to variations that are allowed in the grammar's definition of 'test'. For example, I tend to use 'is None', your example uses 'is not None', so the type checker would have to deal with understanding 'not' in these idioms. Furthermore, there's 'and' and 'or'. When there's a sequence of 'and's, we may assume that each condition is true for the code block that is type-wise affected. When an idiom is found in an 'or', things become complicated even more quickly. Additionally, there are those that use comparison (== 1, > 0) to guage boolean values because of naivite or whatever. Furthermore, there is the issue of applying this check to types other than None. In sum, it is clear that having a type checker understand certain idioms can rapidly become extremely complex and win points with confusing the user all at once. Not a bad recipe for a massively productive faq-populator :) However, this case actually seems quite prevalent to me. default arguments and functions which either return None or something else are quite common. The frequency of usage makes me very much want whatever syntactic construct is put in place to be as similar to existing idiom as possible, both structurally and in terms of ease of use. scott From guido@CNRI.Reston.VA.US Thu Jan 20 03:59:13 2000 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Wed, 19 Jan 2000 22:59:13 -0500 Subject: [Types-sig] question regarding adding words to the grammar In-Reply-To: Your message of "Wed, 19 Jan 2000 19:37:36 EST." <20000119193736.E64311@chronis.pobox.com> References: <20000119193736.E64311@chronis.pobox.com> Message-ID: <200001200359.WAA20832@eric.cnri.reston.va.us> > While I have worked with the pythons grammar to play with type > checking, I found that no matter where a word is introduced in the > grammar, it causes that word to become a keyword, even if that word > cannot appear where a regular variable name might. > > For example, one thing that I once attempted as a way of declaring > classes relationships to interfaces was: > > classdef: class NAME ['(' NAME (',' NAME)* ')'] ['implements' NAME] ':' suite > > However, when I compiled python with this grammar, it wouldn't let me > use the word 'implements' as a variable name (gives SyntaxError). > > So the question is: If you can introduce context-sensitive words, how > do you do it without making a mess out of the grammar and hacking > compile.c? You'd have to hack pgen and friends, the poor excuse for a parser generator that I wrote before starting to write Python, almost exactly ten years ago. Sorry, all quoted identifiers become reserved words. --Guido van Rossum (home page: http://www.python.org/~guido/) From tim_one@email.msn.com Thu Jan 20 05:59:56 2000 From: tim_one@email.msn.com (Tim Peters) Date: Thu, 20 Jan 2000 00:59:56 -0500 Subject: [Types-sig] question regarding adding words to the grammar In-Reply-To: <200001200359.WAA20832@eric.cnri.reston.va.us> Message-ID: <000c01bf630b$93926c20$31a2143f@tim> [scott, wonders how to get context-sensitive literal ids into the grammar without them becoming reserved-everywhere keywords] [Guido] > You'd have to hack pgen and friends, the poor excuse for > a parser generator that I wrote before starting to write > Python, almost exactly ten years ago. I protest! pgen has been solid, effective and convenient from the start. So if that's your idea of a "poor excuse", you should drop Python development at once and make your name writing a *good* parser generator . > Sorry, all quoted identifiers become reserved words. IIRC, the now-defunct "access" stmt had context-sensitive pseudo keywords. Scott, instead of classdef: class NAME ['(' NAME (',' NAME)* ')'] ['implements' NAME] ':' suite I think you'll need to do the more general classdef: class NAME ['(' NAME (',' NAME)* ')'] [NAME NAME] ':' suite and check later that the 4th NAME is "implements". From tim_one@email.msn.com Thu Jan 20 07:13:53 2000 From: tim_one@email.msn.com (Tim Peters) Date: Thu, 20 Jan 2000 02:13:53 -0500 Subject: [Types-sig] Static typing: Towards closure? In-Reply-To: <200001191722.MAA19286@eric.cnri.reston.va.us> Message-ID: <000e01bf6315$e91cb920$31a2143f@tim> [condensing a long & masterful piece to its vital essence here] > I don't like the syntax proposed before very > much; what's wrong with C++ template brackets? In C++, it's the "maximal munch" lexing rule, which tokenizes e.g. Stack> as Stack < Stack < T >> and the result is a mysterious (even after you've been bit a dozen times by the same thing!) syntax error. That's where all the "funny spaces" in C++ templates come from -- to avoid the angle brackets getting parsed as shift operators. Sucks. Other than that, it's fine. That's all from me! I'm recovering from a flu and don't have the time or the functioning brain cells to do much more now. I do want to thank Guido for contributing what has clearly been a great deal of careful thought to all this (he has confessed in the past that he is not a swift writer, so this may have taken him all of a half hour to pump out ). And, of course, by "Guido" I mean Paul and Greg and Scott and Tony and John and Gordon and ... too. Oh ya, also Guido! dreading-the-day-everyone-figures-out-how-hard-this- agenda-really-is<0.7-wink>-ly y'rs - tim From guido@CNRI.Reston.VA.US Thu Jan 20 15:47:03 2000 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Thu, 20 Jan 2000 10:47:03 -0500 Subject: [Types-sig] Static typing: Towards closure? - parameterization Message-ID: <200001201547.KAA21320@eric.cnri.reston.va.us> I believe Josh meant this to go out to the list. ------- Forwarded Message Date: Thu, 20 Jan 2000 09:36:20 -0500 From: josh@shock.pobox.com (Josh Marcus) To: guido@CNRI.Reston.VA.US (Guido van Rossum) Subject: Re: [Types-sig] Static typing: Towards closure? - parameterization > > > While ML has some qualities python has no business pursueing (like how > > it deals with namespaces, currying functions, etc), there is a reason > > it's commonly used to teach about typed programming, the semantics of > > it's type checking system are powerful, concise, and accurate when > > compared to most other languages. I definitely believe it's worth a > > look for the semantics of it's type checking, though not for anything > > else :) > > I have Ullmann's "Elements of ML Programming". Which chapter should I > read? > As I just wrote in an accidentally private note, chapter 12 of "Elements" -- or even the first five or six pages of chapter 12 -- is a good introduction to the ways in which ML's type checking is powerful and clear. (My discussions with Scott about his type checking proposal have turned me into a fan of ML! Who'd of thunk it?) It covers both polymorphic type declarations and constructors, among other things. It's worth noting at this point that ML implements enumerations through the use of datatypes and constructors. While bringing enumerations to python is beyond the scope of type checking, this would be an ideal opportunity to add them, if one was prone to do such a thing. - --j ------- End of Forwarded Message From tismer@tismer.com Thu Jan 20 17:44:11 2000 From: tismer@tismer.com (Christian Tismer) Date: Thu, 20 Jan 2000 18:44:11 +0100 Subject: [Types-sig] Static typing: Towards closure? References: <000e01bf6315$e91cb920$31a2143f@tim> Message-ID: <3887496B.CA9A5E99@tismer.com> Tim Peters wrote: > > [condensing a long & masterful piece to its vital > essence here] > > > I don't like the syntax proposed before very > > much; what's wrong with C++ template brackets? > > In C++, it's the "maximal munch" lexing rule, which tokenizes e.g. > > Stack> > as > Stack > < > Stack > < > T > >> I don't want to add yet another comment on Guido's work, which is great. There is just the "<>" thing that makes me unhappy. If Python adopts the notation of template classes, I get reminded of the fact that the template evaluator in C++ is turing complete! Did you know this? Yes you can compute primes at compile time with templates. Please don't let this happen to Python :-) ciao - chris - voting for smiley templates like Stack(-:T:-) -- Christian Tismer :^) Applied Biometrics GmbH : Have a break! Take a ride on Python's Düppelstr. 31 : *Starship* http://starship.python.net 12163 Berlin : PGP key -> http://wwwkeys.pgp.net PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF we're tired of banana software - shipped green, ripens at home From jeremy@cnri.reston.va.us Thu Jan 20 17:51:01 2000 From: jeremy@cnri.reston.va.us (Jeremy Hylton) Date: Thu, 20 Jan 2000 12:51:01 -0500 (EST) Subject: [Types-sig] Static typing: Towards closure? In-Reply-To: <200001192249.RAA20508@eric.cnri.reston.va.us> References: <20000119171832.B63748@chronis.pobox.com> <200001192249.RAA20508@eric.cnri.reston.va.us> Message-ID: <14471.19205.788587.623442@bitdiddle.cnri.reston.va.us> >>>>> "GvR" == Guido van Rossum writes: [scott writes:] >> *paramterization* your section on parameterization uses the >> example of classes, when use of a type paramater is potentially >> widely available to functions and built in container types like >> lists and tuples. I believe a more general form of >> 'paramterization' which works along the lines of ML's "'a", "'b", >> etc is quite feasible for a static type checking system, as it >> would involve just a few additional checks and operations in the >> most atomic type checking functions. Please take into account in >> the design for paramterization that it should be available for >> more basic types than classes. ML has remarkably clean treatment >> of the idea, btw. GvR> I'm not familiar with ML's 'a, 'b and I don't quite understand GvR> what you are saying here. In the long form of the proposal GvR> there's some text about parameterized functions, e.g.: GvR> def foo(a: T, b: T) -> T: ... The alternate suggestions I made on your whiteboard before the bile/gal bladder digression (note to other readers: don't go there :-) where based on ML's notation. The key difference is that you don't use special angle bracket notation to specify which types are type variables. Instead, you would write: def foo(a: 'a, b: 'a) -> 'a: The ' before the type name would indicate that it is a type variable. In ML, the empty list constructor [] (just like Python!) has the type 'a list. The programmer doesn't need to explicitly declare the type of a list because it will be inferred from the type of the objects that are put in the list. a = [2,3] : int list a.append("foo") # failts because "foo" isn't type int The type checker should infer the most general type, so I assume the following could also be checked: a = [2, 3.5, 2j+1] : number list Jeremy From guido@CNRI.Reston.VA.US Thu Jan 20 18:49:06 2000 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Thu, 20 Jan 2000 13:49:06 -0500 Subject: [Types-sig] Static typing: Towards closure? In-Reply-To: Your message of "Wed, 19 Jan 2000 18:57:42 EST." <20000119185742.C64311@chronis.pobox.com> References: <20000119171832.B63748@chronis.pobox.com> <200001192249.RAA20508@eric.cnri.reston.va.us> <20000119185742.C64311@chronis.pobox.com> Message-ID: <200001201849.NAA21949@eric.cnri.reston.va.us> > From: scott Scott forgot to cc this to the list, so I'm quoting him in its entirety, with my comments interspersed. Don't look for the original message in the archives because it's not there. > On Wed, Jan 19, 2000 at 05:49:53PM -0500, Guido van Rossum wrote: > > > *Constructors* > > > your type proposal seems to mysteriously be missing the notion of > > > type constructors, that is, something like this: > > > > > > datatype FOO = Int | None > > > datatype BAR = Int | None > > > > > > decl a: FOO > > > decl b: BAR > > > > > > a = 1 > > > b = a # ERROR! > > > > Actually, in my mind, this would be allowed. I believe in structural > > type matching, not matching by name. (That's what I've been > > believing, anyway.) Possibly when you introduce classes there's no > > me too, in the general case. i think most type checking in python > should be structural, but it is undeniable that allowing the user to > produce completely new datatypes (rather than just compound ones) > can produce safer code: > > datatype FIRSTNAME -> FIRSTNAME = string > datatype LASTNAME -> LASTNAME = string > datatype NICKNAME -> NICKNAME = string > > def get_nickname_from_firstname(n: FIRSTNAME) -> NICKNAME: > # do stuff > > def get_nickname_from_lastname(n: LASTNAME) -> NICKNAME: > # do stuff > > > # To use this, you need a constructor in order to produce objects > # with the new datatypes: > > decl a: FIRSTNAME > a = FIRSTNAME("scott") > print get_nickname_from_lastname(a) # ERROR > > By requiring the casts, there is also a good argument that it makes > code more readable. > > Personally, I wouldn't use this much in python even if it were > available, I'd opt for 'structural' types 95% of the time. But I'd > appreciate it being available, and recognize its utility in > appropriate circumstances. It's mostly just something that comes from > examining what's out there and why it's useful. I think it's a nicety that's best left out of our initial design. As you say, it's not needed in 95% of the time. And Python is kind of a 95% language... :-) > > way to have another class match even if it has the same structure, > > but for simple unions I don't see the need. (Or is your "datatype" a > > special declaration different from my typedef, which I spell > > > > decl FOO = int | None > > > > ?) > > > > I believe the notion of 'datatype' could be put to many uses, defining > types in many different ways. One could use another term, or use lots > of terms, but I am working under the assumption that lots of keywords > are bad for compatibility reasons. I currently (ab)use the decl keyword both for defining new types and for declaring the types of objects; decl T = S defines a new type T to be a new name for type S (which could be a type expression), while decl v: T declares a variable v of type T. This uses only one keyword and is unambiguous. Can't get better than that. :-) > > > b = BAR(a) # ok, explicit cast or 'constructor' > > > > > > It is possible to parse the cast at compile time, since it requires > > > no knowledge of the value of `a', only its type. > > > > > > The above example is slightly contrary to a grammar I put together, > > > and I don't believe it's the right way to denote constructors, but it > > > does demonstrate the intended semantics. > > > > > > using constructors can make dealing with logically OR'd types much > > > easier and cleaner, and allows the user to define more specific types > > > than allowed with builtin types. I think it's quite a worthwhile pursuit. > > > > I have chosen names for the std types that are the same as existing > > built-in functions, because I want to get a similar effect. E.g. > > > > x = int(y) > > > > is currently a call of the built-in function int() which happens to > > return its argument converted to an int, or die with an exception; in > > the new system, it would have the same effect but it would be a > > constructor for the int datatype (and the built-in object int would be > > the same object as types.IntType). > > There is a semantic distinction between using 'int' as both a > constructor and a type and the above example: 'int' the builtin > function alters the value of its argument. In the above, there's no > mechanism for doing that -- it's only the type that changes. > Nonetheless the similarity between the two is curious, and has > potential for some nice continuity with the builtin typecast > functions. Of course, the builtin function doesn't alter the argument (how could it -- it's probably an immutable object like a string) but I understand what you mean: the returned object is not always the same object as the argument. This is following the example of C, where (int) and (float) casts can change the value, in a sense, while typical pointer casts only change the type. This may be seen as a bad example, but I'm not so sure. I think it's possible to live with the slight conceptual mismatch. Maybe a more serious problem is whether this extends to classes: if C is a class, and D is a derived class, we might want to be able to cast between the two using the same notation: decl myC: C, myD: D myC = C(myD) # downcast myD = D(myC) # upcast; runtime typecheck inserted But the problem here is that C(x) and D(x) are already constructor calls and may have very different meanings... > > It's true that these constructors have something of a dynamic cast in > > them (they can fail if the argument is unacceptable). I think that in > > my proposal, > > > > x = int(a) > > > > would have about the same semantics as Greg's > > > > x = a ! int > > The user defined casts wouldn't suffer from that problem: they don't > alter any values, just the associated types of the values. What do you call a user defined cast? --Guido van Rossum (home page: http://www.python.org/~guido/) From faassen@vet.uu.nl Thu Jan 20 22:16:25 2000 From: faassen@vet.uu.nl (Martijn Faassen) Date: Thu, 20 Jan 2000 23:16:25 +0100 Subject: [Types-sig] Static typing: Towards closure? In-Reply-To: <200001191722.MAA19286@eric.cnri.reston.va.us> References: <200001191722.MAA19286@eric.cnri.reston.va.us> Message-ID: <20000120231625.A899@vet.uu.nl> [snip interesting proposal doc] A few points that I didn't see discussed by Guido in his proposal document. They range from a syntactic nit to a discussion on interface semantics, so please bear with me. :) I) Beware of overloading ':' too much. The colon currently I think has three meanings already, luckily fairly easily distinguishable by the eye: * Block introducer class Foo: ... def Bar: ... * slice syntax a[1:10] * dictionary syntax a = {"foo" : "bar"} I don't think the type annotation use is that easy on the eyeballs, though: decl foo: integer It's too much like def foo: integer Of course the latter doesn't mean anything, but it's still too visually similar to block introducer use to me. We may therefore want to use something other than the colon for type annotations. Just to be contrary again, I'll again advocate the use of 'the' (inspired by Tim Peters): decl foo the integer def foo(a the string, b the Bar) -> integer: pass II) The stubbing problem. Currently, Python allows this: def foo(a, b): pass Nobody complains, we can run and test our code. This is good for rapid prototyping. However: def foo(a, b) -> integer: pass will presumabily cause the typechecker to barf. I'm not quite sure if this is really bad, but I just wanted to bring it to people's attention. III) My previous point implies in my mind an important principle: When adding type checking to Python, the speed of developing in Python should *not* go away. If static type checking makes us slow down too much, we will have failed. This splits into two requirements: * We want to keep Python as it is today. We want to avoid the chance that static types run amok, subtly or non-subtly pushing people to add them everywhere, as this could slow down development a lot. * We want statically checked Python to be as much as possible like Python today in regards to quick development time. Ideally Python with type annotations should not be that much slower to develop in than Python without. My next point is influenced by this important principle as well. IV) interfaces As I remarked earlier on this list, perhaps it's useful to think of all types as interfaces. The type of an instance is the interface type defined by its class. The interfaces form an inheritance tree of the form of a DAG. A class always implies at least one interface; that of the same name as the classes name. [If two classess exist in a namespace with the same name, this is only valid if their interfaces are structurally identical (or perhaps use of them is only valid if only the structurally identical part of the interface is used).] Interfaces can be implied: class Foo implements Alpha: decl greeting: integer def hey(a: integer)->integer: ... def hi(a: string)->string: ... def hoi(a: float)->float: ... class Bar implements Alpha: decl greeting: integer decl specialgreeting: integer def hey(a: integer)->integer: ... def hi(a: string)->string: ... Alpha is now deduced to be the following interface: interface Alpha: decl greeting: integer def hey(a: integer)->integer: ... def hi(a: string)->string: ... Note that you don't need to write this down anywhere! It's implicit (until explicitized by the developer). The consequences: * no need to explicitly code up interfaces, so existing Python code is more easily brought into a typecheckable form. You could for instance say each FileObject type class implements the interface FileObject. * no need for access specifiers; private members are implied as they usually won't appear in both classes, and are thus not part of the interface. * interfaces are easily shrunken if necessary, by declaring a special class with the maximum interface. * interfaces can always be made explicit (and then any class that states it conforms to it but doesn't is in error), so that a minimal interface can be specified. These consequences may be advantages during development; interfaces (or types) can be quickly and conveniently created and used, and they are always _right_; we're not bothered by interface compliance errors as the interfaces are implied. This is good as early in development we may still be prototyping, and therefore we may not be clear on the exact structure of the interfaces yet. We don't want to go back and forth to change the interface definition all the time as we discover new methods are needed, or some should be removed. We only have to edit the actual classes implementing the interface. As the code solidifies we may want to do more checking, so we can start to explicitize the interfaces. This is similar to the pattern we're hoping to accomplish by adding static typechecking -- it should be possible to code in the dynamic way and then tighten up code by adding type annotations later. I posted earlier to the list on this idea, but either nobody read it, nobody understood it or everybody thought it was obviously a bad idea. I'd like to be informed on which is the right explanation. :) Regards, Martijn From guido@CNRI.Reston.VA.US Thu Jan 20 22:31:04 2000 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Thu, 20 Jan 2000 17:31:04 -0500 Subject: [Types-sig] Static typing: Towards closure? In-Reply-To: Your message of "Thu, 20 Jan 2000 23:16:25 +0100." <20000120231625.A899@vet.uu.nl> References: <200001191722.MAA19286@eric.cnri.reston.va.us> <20000120231625.A899@vet.uu.nl> Message-ID: <200001202231.RAA27334@eric.cnri.reston.va.us> > I) Beware of overloading ':' too much. Point taken. > decl foo the integer > > def foo(a the string, b the Bar) -> integer: > pass I don't like this; the colon still looks best to me. > II) The stubbing problem. > def foo(a, b) -> integer: > pass > > will presumabily cause the typechecker to barf. I'm not quite sure if this > is really bad, but I just wanted to bring it to people's attention. Do this the same say as in Java or C++: return a dummy value. > III) > > My previous point implies in my mind an important principle: > > When adding type checking to Python, the speed of developing in Python should > *not* go away. If static type checking makes us slow down too much, we > will have failed. > > This splits into two requirements: > > * We want to keep Python as it is today. We want to avoid the chance that > static types run amok, subtly or non-subtly pushing people to add them > everywhere, as this could slow down development a lot. Point taken (Skip also warned about this). > * We want statically checked Python to be as much as possible like Python > today in regards to quick development time. Ideally Python with type > annotations should not be that much slower to develop in than Python without. Actually, we would expect it to be faster, because errors are caught earlier (during code generation rather than during testing or when in production) and this should save time. Of course there's more typing (I'd say about 10% more). > My next point is influenced by this important principle as well. > > IV) interfaces > Interfaces can be implied: > > class Foo implements Alpha: > decl greeting: integer > > def hey(a: integer)->integer: > ... > > def hi(a: string)->string: > ... > > def hoi(a: float)->float: > ... > > class Bar implements Alpha: > > decl greeting: integer > decl specialgreeting: integer > > def hey(a: integer)->integer: > ... > > def hi(a: string)->string: > ... > > Alpha is now deduced to be the following interface: > > interface Alpha: > decl greeting: integer > > def hey(a: integer)->integer: > ... > > def hi(a: string)->string: > ... > > Note that you don't need to write this down anywhere! It's implicit > (until explicitized by the developer). Urk! This defeats the purpose of static checking; it's no better than not using static checking at all! > The consequences: > > * no need to explicitly code up interfaces, so existing Python code is > more easily brought into a typecheckable form. You could for instance > say each FileObject type class implements the interface FileObject. There are lots of subtleties, and I *like* the idea of an explicitly specified interface to disambiguate what is a file (e.g. must it have seek()? fileno()? isatty()?). > * no need for access specifiers; private members are implied as they > usually won't appear in both classes, and are thus not part of the > interface. Waiting for accidents to happen (unrelated variables that happen to have the same name). > * interfaces are easily shrunken if necessary, by declaring a special class > with the maximum interface. > > * interfaces can always be made explicit (and then any class that states it > conforms to it but doesn't is in error), so that a minimal interface can > be specified. > > These consequences may be advantages during development; interfaces (or types) > can be quickly and conveniently created and used, and they are always > _right_; we're not bothered by interface compliance errors as the interfaces > are implied. > > This is good as early in development we may still be prototyping, and > therefore we may not be clear on the exact structure of the interfaces > yet. We don't want to go back and forth to change the interface definition > all the time as we discover new methods are needed, or some should be removed. > We only have to edit the actual classes implementing the interface. > > As the code solidifies we may want to do more checking, so we > can start to explicitize the interfaces. This is similar to the pattern we're > hoping to accomplish by adding static typechecking -- it should be possible > to code in the dynamic way and then tighten up code by adding type annotations > later. It sounds like a much better thing to do is to leave out all type declarations during prototyping, and add them as you start agreeing on what the interfaces should be. > I posted earlier to the list on this idea, but either nobody read it, > nobody understood it or everybody thought it was obviously a bad idea. I'd > like to be informed on which is the right explanation. :) My-guess-is-it-was-obvious-ly y'rs, --Guido van Rossum (home page: http://www.python.org/~guido/) From da@ski.org Thu Jan 20 22:39:23 2000 From: da@ski.org (David Ascher) Date: Thu, 20 Jan 2000 14:39:23 -0800 Subject: [Types-sig] Static typing: Towards closure? In-Reply-To: <20000120231625.A899@vet.uu.nl> Message-ID: <001f01bf6397$32bad5e0$c355cfc0@ski.org> Martijn Faassen: > Interfaces can be implied: > > class Foo implements Alpha: [...] > class Bar implements Alpha: [..] > Alpha is now deduced to be the following interface: This is very odd, IMO. I'd expect Python not to *deduce* the interface but to *check* that the interface is indeed being followed. > These consequences may be advantages during development; > interfaces (or types) can be quickly and conveniently created and used, > and they are always _right_; we're not bothered by interface compliance > errors as the interfaces are implied. How does the deduced interface get communicated to the programmer, and/or used? class MyFileObject implements Stream: def read_bytes(numBytes : integer) -> string class MyOtherFileObject implements Stream: def readBytes(numBytes : integer) -> string leads to an empty interface Stream, due to a typo. What are the semantic consequences of the implements statement? I'm missing the point, I fear. > I posted earlier to the list on this idea, but either nobody read it, > nobody understood it or everybody thought it was obviously a bad idea. I'd > like to be informed on which is the right explanation. :) First time around, i didn't read it. This time, I don't understand it. =) -david From faassen@vet.uu.nl Thu Jan 20 22:49:12 2000 From: faassen@vet.uu.nl (Martijn Faassen) Date: Thu, 20 Jan 2000 23:49:12 +0100 Subject: [Types-sig] Static typing: Towards closure? In-Reply-To: <001f01bf6397$32bad5e0$c355cfc0@ski.org> References: <20000120231625.A899@vet.uu.nl> <001f01bf6397$32bad5e0$c355cfc0@ski.org> Message-ID: <20000120234912.A987@vet.uu.nl> David Ascher wrote: > Martijn Faassen: > > > Interfaces can be implied: > > > > class Foo implements Alpha: > [...] > > class Bar implements Alpha: > [..] > > Alpha is now deduced to be the following interface: > > This is very odd, IMO. I'd expect Python not to *deduce* the interface but > to *check* that the interface is indeed being followed. Yes, check when the interface is *used*: def myfunc(a: Alpha): ... If a isn't of class Foo or Bar in this case, there'd be a compile-time error. If you used anything on a that is not part of the intersection of the interfaces of class Foo and Bar, you'd also get a compile-time error. > > These consequences may be advantages during development; > > interfaces (or types) can be quickly and conveniently created and used, > > and they are always _right_; we're not bothered by interface compliance > > errors as the interfaces are implied. > > How does the deduced interface get communicated to the programmer, and/or > used? > > class MyFileObject implements Stream: > def read_bytes(numBytes : integer) -> string > > class MyOtherFileObject implements Stream: > def readBytes(numBytes : integer) -> string > > leads to an empty interface Stream, due to a typo. Empty interfaces should of course be detected by the compiler and complained about. Though you may be right in that a typo could screw up things -- if the intersection of MyFileObject and MyOtherFileObject is not empty but misses a crucial method due to a typo, the compiler will complain when an attempt is made to use either of the methods involved. You'd have to go through all classes implementing the interface in order to fix the problem. That's Not Nice and probably destroys my idea, unless I can figure out how to neutralize this problem. Explicit interfaces don't have this problem. > What are the semantic consequences of the implements statement? If the interface (Stream) is not explicitly defined somewhere, it'll be the intersection of the interfaces of the classes MyFileObject and MyOtherFileObject. > I'm missing the point, I fear. [snip] The point is the intersection concept. Does my previous post make more sense with this new information? Regards, Martijn From da@ski.org Thu Jan 20 23:07:32 2000 From: da@ski.org (David Ascher) Date: Thu, 20 Jan 2000 15:07:32 -0800 Subject: [Types-sig] Static typing: Towards closure? In-Reply-To: <20000120234912.A987@vet.uu.nl> Message-ID: <002001bf639b$21b230f0$c355cfc0@ski.org> > From: Martijn Faassen [mailto:faassen@vet.uu.nl] > If you used anything on a that is not part of the intersection of the > interfaces of class Foo and Bar, you'd also get a compile-time error. > Empty interfaces should of course be detected by the compiler and > complained about. That was an extreme example. In general there is no way for the programmer to make sure that s/he didn't put in a typo or inadvertently broaden the interface simply because all implementations of that interface share an attribute name. That's exactly where a typechecking system should help! > The point is the intersection concept. Does my previous post make more > sense with this new information? I think I understand what you're trying to propose. I also think it's much too weak. I much prefer explicit interface checking. Design by Contract is a good idea only if the Contract is explicit. Implicit agreements in business lead to lawsuits and unhappiness... Anyway, interfaces are off the table, remember? --david From faassen@vet.uu.nl Thu Jan 20 23:41:12 2000 From: faassen@vet.uu.nl (Martijn Faassen) Date: Fri, 21 Jan 2000 00:41:12 +0100 Subject: [Types-sig] Static typing: Towards closure? In-Reply-To: <200001202231.RAA27334@eric.cnri.reston.va.us> References: <200001191722.MAA19286@eric.cnri.reston.va.us> <20000120231625.A899@vet.uu.nl> <200001202231.RAA27334@eric.cnri.reston.va.us> Message-ID: <20000121004112.B987@vet.uu.nl> Guido van Rossum wrote: ['the' proposal] > I don't like this; the colon still looks best to me. This may be because the colon is common in other languages for this purpose (and shorter) -- but I am still worried it may not be good in Python. > > II) The stubbing problem. > > > def foo(a, b) -> integer: > > pass > > > > will presumabily cause the typechecker to barf. I'm not quite sure if this > > is really bad, but I just wanted to bring it to people's attention. > > Do this the same say as in Java or C++: return a dummy value. Yes, I thought of that, and it may be enough. I just thought to bring the breakdown of this common usage to people's attention. :) [snip] > > * We want statically checked Python to be as much as possible like Python > > today in regards to quick development time. Ideally Python with type > > annotations should not be that much slower to develop in than Python without. > > Actually, we would expect it to be faster, because errors are caught > earlier (during code generation rather than during testing or when in > production) and this should save time. Hm, no. If this were so we'd all be using Java or C++. Catching errors early is good in large systems for maintainability reasons, and in the situation where the approach to solving a problem is thoroughly understood. Often while coding you are prototyping however; parts of the code don't make sense and this is known, but we want a quick way to test parts of the code that already works. This is tolerated by Python currently partly because it doesn't do type checking. You don't have to think about having to have all the interfaces right and all the types right in the beginning. This can speed up development in Python, where in C++ I get bogged down in nailing down lots of class interfaces before I can proceed to the meat of the problem (where I may find out my initial approach is all wrong and I did a lot of work for nothing). If type-annotated Python is infecteous (Skip I think called this the 'const' propagation problem, like the way you need to const-correct everything in C++ if you use const at all), we have a problem. I'm still worried we'll end up with a language where there's a type-propagation problem; if we use type annotations somewhere, we may have to add them everywhere. My worries may not be justified at all; I suppose we'll have to have some practical experience with it before we can really know. > Of course there's more typing (I'd say about 10% more). And more thinking, we want to know how many % more that will be too. :) [idea: an interface is automatically deduced from the intersection of two or more class interfaces] > > Note that you don't need to write this down anywhere! It's implicit > > (until explicitized by the developer). > > Urk! This defeats the purpose of static checking; it's no better than > not using static checking at all! Why? Static type checking is still occurring: class Foo implements Alpha: def a(self): pass def b(self): pass class Bar implements Alpha: def a(self): pass def c(self): pass def myfunc(obj: Alpha): obj.a() # this works obj.b() # this doesn't work -- compile time rejection obj.c() # this doesn't work either -- compile time rejection A problem that _does_ occur with this scheme is that if you get a compile time rejection due to a typo in a method name (for instance), and suddenly that method is not part of the interface while it should be, you may not know which class implementing the interface has the actual typo, something you don't have with explicit interfaces. This problem may be fatal. But perhaps smart error reporting could help: method b() does not exist on interface Alpha. classes that do have it are: Foo, Baz classes that don't have it are: Baz method boo() does not exist on interface Alpha nor on any classes that implement it. Another option is to allow partially explicit interfaces. You notice somehow your implicit interface Alpha doesn't have the method 'a'. You want it. You say: partial Alpha: def a(self): pass Now the compiler will complain as soon as a class that claims to implement Alpha does not in fact define the method named 'a'. Later on you can grow a partial interface to a full one, if desired. > > The consequences: > > > > * no need to explicitly code up interfaces, so existing Python code is > > more easily brought into a typecheckable form. You could for instance > > say each FileObject type class implements the interface FileObject. > > There are lots of subtleties, Of course. I just wanted to see some discussion on the idea. > and I *like* the idea of an explicitly > specified interface to disambiguate what is a file (e.g. must it have > seek()? fileno()? isatty()?). Me too, but you can of course always explicitly disambiguate it by writing down an explicit interface. I don't want to take away explicit interfaces at all. They're good. > > * no need for access specifiers; private members are implied as they > > usually won't appear in both classes, and are thus not part of the > > interface. > > Waiting for accidents to happen (unrelated variables that happen to > have the same name). I realize this, but is it that disastrous? Current dynamically typed Python has the exact same problem. [snip] > > As the code solidifies we may want to do more checking, so we > > can start to explicitize the interfaces. This is similar to the pattern we're > > hoping to accomplish by adding static typechecking -- it should be possible > > to code in the dynamic way and then tighten up code by adding type annotations > > later. > > It sounds like a much better thing to do is to leave out all type > declarations during prototyping, and add them as you start agreeing on > what the interfaces should be. You'd have to leave out all type declarations in the code that *uses* instances of your classes, then. class Foo: def a(self): pass class Bar: def a(self): pass def b(self): pass # this doesn't seem to work right def use1(obj: Foo | Bar): obj.b() # what happens here? Is this allowed? or is an intersection taken? obj = Foo() use1(obj) # this gives a compile time error but we can't pass in any instances of # Bar. def use2(obj: Foo): obj.b() # this causes the error # so we have to do this in order to be able to use both Foo and Bar here: def use3(obj): obj.b() # run-time error! # now, if Foo and Bar both stated they implemented the interface Alpha, # the compiler could deal with this: def use4(obj: Alpha): obj.b() # compile time error Isn't this a good intermediate step between having an explicit interface and not having any interface at all? [note: it would also be possible to give Foo and Bar a common baseclass and require that type to be used: def use5(obj: Common): obj.b() # compile time error, not defined by Common but that essentially raises the same objections as having to explicitize interfaces early] > > I posted earlier to the list on this idea, but either nobody read it, > > nobody understood it or everybody thought it was obviously a bad idea. I'd > > like to be informed on which is the right explanation. :) > > My-guess-is-it-was-obvious-ly y'rs, Not-afraid-to-argue-about-obviously-bad-ideas-ly yours, Martijn From faassen@vet.uu.nl Thu Jan 20 23:46:43 2000 From: faassen@vet.uu.nl (Martijn Faassen) Date: Fri, 21 Jan 2000 00:46:43 +0100 Subject: [Types-sig] Static typing: Towards closure? In-Reply-To: <002001bf639b$21b230f0$c355cfc0@ski.org> References: <20000120234912.A987@vet.uu.nl> <002001bf639b$21b230f0$c355cfc0@ski.org> Message-ID: <20000121004643.A1050@vet.uu.nl> David Ascher wrote: > > interfaces of class Foo and Bar, you'd also get a compile-time error. > > > Empty interfaces should of course be detected by the compiler and > > complained about. > > That was an extreme example. In general there is no way for the programmer > to make sure that s/he didn't put in a typo or inadvertently broaden the > interface simply because all implementations of that interface share an > attribute name. That's exactly where a typechecking system should help! And it still will, as soon as you explicitize the interface. I'm proposing an intermediate stage. After all we live without explicit interfaces in Python right now -- an intermediate stage *with* couldn't be _worse_, right? :) (yeah, yeah, I realize it could be, but realize where we are coming from). > > The point is the intersection concept. Does my previous post make more > > sense with this new information? > > I think I understand what you're trying to propose. I also think it's much > too weak. I much prefer explicit interface checking. Design by Contract is > a good idea only if the Contract is explicit. Implicit agreements in > business lead to lawsuits and unhappiness... But interfaces can currently be completely implicit! I'm actually *adding* explicitness to the current system. :) > Anyway, interfaces are off the table, remember? Yes, but several people did put them on the table again in previous discussions on the SIG; it was realized it's hard to keep them off the table and still have a fruitful discussion (and by several people I mean Paul, and also I believe Greg and Guido). Devil's-advocate-ly yours, Martijn From tim_one@email.msn.com Thu Jan 20 23:53:15 2000 From: tim_one@email.msn.com (Tim Peters) Date: Thu, 20 Jan 2000 18:53:15 -0500 Subject: [Types-sig] Static typing: Towards closure? In-Reply-To: <20000120231625.A899@vet.uu.nl> Message-ID: <000101bf63a1$849f9a80$6ea0143f@tim> [Martijn Faassen] > ... > I don't think the type annotation use is that easy on the > eyeballs, though: > > decl foo: integer > > It's too much like > > def foo: integer > > Of course the latter doesn't mean anything, but it's > still too visually similar to block introducer use > to me. Except these don't look like blocks to me at all. def foo: integer looks like a block. That is, a colon that introduces a block is usually the last significant character on a line (always, for people like me). > We may therefore want to use something other than the > colon for type annotations. Just to be contrary again, > I'll again advocate the use of 'the' (inspired by > Tim Peters): > > decl foo the integer Yoda the Jedi Master, strangely he speaks . "the" is a function from Common Lisp, where the ordering reads much better, as in (the integer 42) It's CL's prefix version of Greg's infix "!" runtime function. While I have no objection to a colon here, "is" would read a lot better than "the" if most others do: decl foo is integer As to speed of development, I'm with Skip in believing static declarations must never be *required* -- unless you've explicitly asked for "decl" mode. If I get significant ERR or OPT or DOC benefits from explicit typing (& I have no reason to use it at all if I don't!), of course I'm going to spend more time typing to get that. Note that type decls are also subject to user mistakes, and 10% of the lines are decls" doesn't imply a proportional amount of effort. Where static typing costs in development time way out of proportion to its line count is under modification, where "all of a sudden" a conceptually simple change of type in one place ends up getting manually propagated all over the place (method X used to return Y but now returns Z, so potentially all call sites need to get redeclared, as well as potentially all code that-- directly or indirectly -- "consumes" the return value(s)). What is a one-line change in untyped Python may require an unbounded number of changes in typed Python. premature-static-typing-is-the-root-of-all-evil-ly y'rs - tim From jeremy@cnri.reston.va.us Thu Jan 20 23:56:57 2000 From: jeremy@cnri.reston.va.us (Jeremy Hylton) Date: Thu, 20 Jan 2000 18:56:57 -0500 (EST) Subject: [Types-sig] Static typing: Towards closure? In-Reply-To: <20000121004112.B987@vet.uu.nl> References: <200001191722.MAA19286@eric.cnri.reston.va.us> <20000120231625.A899@vet.uu.nl> <200001202231.RAA27334@eric.cnri.reston.va.us> <20000121004112.B987@vet.uu.nl> Message-ID: <14471.41161.944820.970237@bitdiddle.cnri.reston.va.us> >>>>> "MF" == Martijn Faassen writes: MF> If type-annotated Python is infecteous (Skip I think called this MF> the 'const' propagation problem, like the way you need to MF> const-correct everything in C++ if you use const at all), we MF> have a problem. I'm still worried we'll end up with a language MF> where there's a type-propagation problem; if we use type MF> annotations somewhere, we may have to add them everywhere. My MF> worries may not be justified at all; I suppose we'll have to MF> have some practical experience with it before we can really MF> know. A related worry is that the type checking will have unexpected effects on performance when there is an interaction between checked and unchecked code. Guido showed the potentially painful example of passing a list containing thousands of elements from unchecked code to an object declared as "def (int list) -> int"; this seems to imply a runtime check to verify that all of the elements of the list are ints. I worry that there would be subtler cases with similar performance hits. Jeremy From guido@CNRI.Reston.VA.US Fri Jan 21 00:01:07 2000 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Thu, 20 Jan 2000 19:01:07 -0500 Subject: [Types-sig] Static typing: Towards closure? In-Reply-To: Your message of "Fri, 21 Jan 2000 00:41:12 +0100." <20000121004112.B987@vet.uu.nl> References: <200001191722.MAA19286@eric.cnri.reston.va.us> <20000120231625.A899@vet.uu.nl> <200001202231.RAA27334@eric.cnri.reston.va.us> <20000121004112.B987@vet.uu.nl> Message-ID: <200001210001.TAA28120@eric.cnri.reston.va.us> Martijn, I won't discuss the implied interfaces idea any further on the list. I just find it too absurd. Since you're coming to the conference next week, I'll buy you a beer and we can talk about it under more enlightened circumstances. Maybe I'll even remeber a few words of Dutch! :-) Other stuff: > > Actually, we would expect it to be faster, because errors are caught > > earlier (during code generation rather than during testing or when in > > production) and this should save time. > > Hm, no. If this were so we'd all be using Java or C++. Touche. > Catching errors early is good in large systems for maintainability reasons, > and in the situation where the approach to solving a problem is thoroughly > understood. Often while coding you are prototyping however; parts of the > code don't make sense and this is known, but we want a quick way to > test parts of the code that already works. > > This is tolerated by Python currently partly because it doesn't do type > checking. You don't have to think about having to have all the interfaces > right and all the types right in the beginning. This can speed up > development in Python, where in C++ I get bogged down in nailing down lots > of class interfaces before I can proceed to the meat of the problem (where > I may find out my initial approach is all wrong and I did a lot of work > for nothing). > > If type-annotated Python is infecteous (Skip I think called this the 'const' > propagation problem, like the way you need to const-correct everything in C++ > if you use const at all), we have a problem. I'm still worried we'll end > up with a language where there's a type-propagation problem; if we use > type annotations somewhere, we may have to add them everywhere. My worries > may not be justified at all; I suppose we'll have to have some practical > experience with it before we can really know. I don't expect that there will be a propagation problem, even if all the standard library modules (and extensions) are typechecked. My reasoning is that the type system only adds static warnings (i.e., restrictions) to checked modules; unchecked modules (such as you are likely to be using during prototyping) will only get runtime errors, and they should only get runtime errors where they would have gotten runtime errors before anyway. There are two exceptions: maybe you'll get warnings when you compile an unchecked module that imports a checked module and then uses it in an obviously broken way (e.g. calling a function that's not defined or calling a function that is defined with the wrong number of arguments); but that's code that's already broken. And maybe code that abuses the standard library will break (e.g. if you put things in sys.modules that aren't modules or whose keys aren't strings). But calling a standard function that takes an int with an object whose type is only known as 'any' at compile time is not a compile-time error; you only get an error when at runtime it's not an int, and that's no different than before (except that sometimes the diagnostic will be more understandable; currently you sometimes get these errors after the illegal value has been passed down several levels deep into the internals of a library module). But I don't think this will affect your ability to write unchecked modules at all. The 'const' example doesn't apply here. --Guido van Rossum (home page: http://www.python.org/~guido/) From faassen@vet.uu.nl Fri Jan 21 00:02:24 2000 From: faassen@vet.uu.nl (Martijn Faassen) Date: Fri, 21 Jan 2000 01:02:24 +0100 Subject: [Types-sig] Static typing: Towards closure? In-Reply-To: <000101bf63a1$849f9a80$6ea0143f@tim> References: <20000120231625.A899@vet.uu.nl> <000101bf63a1$849f9a80$6ea0143f@tim> Message-ID: <20000121010224.A1089@vet.uu.nl> Tim Peters wrote: [I say def foo: integer is too much like decl foo: integer] > Except these don't look like blocks to me at all. > > def foo: > integer > > looks like a block. That is, a colon that introduces a block is usually the > last significant character on a line (always, for people like me). I write code like that too, but unfortunately not everybody does... [snip Tim (Yoda?) discussing 'the'] [static types shouldn't be required; 10% extra lines doesn't necessarily mean 10% extra effort, type change propagation difficulties causing lots of reworking of code] > premature-static-typing-is-the-root-of-all-evil-ly Agreed -- you expressed it better than I could. I don't want to end up not daring to change anything, because types could break, and I don't want to end up having to engineer complicated interfaces before I can actually test them. In C++ (the statically typed language I'm most familiar with) you can easily end up in just this situation. Regards, Martijn From paul@prescod.net Thu Jan 20 23:58:54 2000 From: paul@prescod.net (Paul Prescod) Date: Thu, 20 Jan 2000 15:58:54 -0800 Subject: [Types-sig] Static typing: Towards closure? References: <200001191722.MAA19286@eric.cnri.reston.va.us> <20000120231625.A899@vet.uu.nl> <200001202231.RAA27334@eric.cnri.reston.va.us> <20000121004112.B987@vet.uu.nl> Message-ID: <3887A13E.36E91B2F@prescod.net> Martijn Faassen wrote: > > .... > If type-annotated Python is infecteous (Skip I think called this the 'const' > propagation problem, like the way you need to const-correct everything in C++ > if you use const at all), we have a problem. We started out with the lack of infectiousness as one of the most immutable design goal. Achieving this is really not hard at all. When you call from Python (dynamically typed) into C or Java (statically typed) you don't have to add type declarations to your Python code, do you? So why would you need to add them if you call from dynamically typed Python into statically typed Python. "const" is infectious in C++ because that's the whole point. It was designed to be infectious. It would have been useless if it were not infectious. > I'm still worried we'll end > up with a language where there's a type-propagation problem; if we use > type annotations somewhere, we may have to add them everywhere. My worries > may not be justified at all; I suppose we'll have to have some practical > experience with it before we can really know. We do have experience and we do know. Use JPython, for example. -- Paul Prescod - ISOGEN Consulting Engineer speaking for himself Earth will soon support only survivor species -- dandelions, roaches, lizards, thistles, crows, rats. Not to mention 10 billion humans. - Planet of the Weeds, Harper's Magazine, October 1998 From faassen@vet.uu.nl Fri Jan 21 00:38:12 2000 From: faassen@vet.uu.nl (Martijn Faassen) Date: Fri, 21 Jan 2000 01:38:12 +0100 Subject: [Types-sig] Static typing: Towards closure? In-Reply-To: <200001210001.TAA28120@eric.cnri.reston.va.us> References: <200001191722.MAA19286@eric.cnri.reston.va.us> <20000120231625.A899@vet.uu.nl> <200001202231.RAA27334@eric.cnri.reston.va.us> <20000121004112.B987@vet.uu.nl> <200001210001.TAA28120@eric.cnri.reston.va.us> Message-ID: <20000121013812.A1120@vet.uu.nl> Guido van Rossum wrote: > Martijn, I won't discuss the implied interfaces idea any further on > the list. I just find it too absurd. Okay, then I won't point out that currently Python allows interfaces that are even more implicit. :) [whoops, sorry, I'll shut up now] > Since you're coming to the > conference next week, I'll buy you a beer and we can talk about it > under more enlightened circumstances. Maybe I'll even remeber a few > words of Dutch! :-) Sounds good to me! :) Sorry to play devil's advocate so persistently; it's one of my life's tenets that even absurd ideas have merit in training the mind and gaining new insights -- which is why I end up talking in absurdities quite frequently (usually to hopefully comic effect). Anyway, a beer, good! [snip] > > If type-annotated Python is infecteous (Skip I think called this the 'const' > > propagation problem, like the way you need to const-correct everything in C++ > > if you use const at all), we have a problem. I'm still worried we'll end > > up with a language where there's a type-propagation problem; if we use > > type annotations somewhere, we may have to add them everywhere. My worries > > may not be justified at all; I suppose we'll have to have some practical > > experience with it before we can really know. > > I don't expect that there will be a propagation problem, even if all > the standard library modules (and extensions) are typechecked. My > reasoning is that the type system only adds static warnings (i.e., > restrictions) to checked modules; unchecked modules (such as you are > likely to be using during prototyping) will only get runtime errors, > and they should only get runtime errors where they would have gotten > runtime errors before anyway. I hadn't considered this distinction between checked and unchecked fully enough; I'll have to think this one over. Can an unchecked module have type annotations too, and can this cause run-time TypeError exceptions to be raised? Perhaps this is in your document and I missed it. Note that runtime checking might help avoid the list of 1000 ints problem in some cases; this list can be declared to be of ints and this can be checked at runtime, so that passing it into checked function can proceed without overhead. But, if this kind of thing is allowed, you could very well have a type-infection problem; in order to avoid the overhead people are pushed to add checks everywhere. [snip] > But I don't think this will affect your ability to write unchecked > modules at all. The 'const' example doesn't apply here. I agree, the const example doesn't apply directly, but I was thinking more about the more subtle cultural thing that makes C++ programmers want to const-correct everything -- I don't think we want a similar drive to 'type-correctness' in Python. Anyway, we won't know until we try, so I'll stop arguing about this. For-now-at-least-ly yours, Martijn From paul@prescod.net Fri Jan 21 00:09:32 2000 From: paul@prescod.net (Paul Prescod) Date: Thu, 20 Jan 2000 16:09:32 -0800 Subject: [Types-sig] Static typing: Towards closure? References: <200001191722.MAA19286@eric.cnri.reston.va.us> <20000120231625.A899@vet.uu.nl> <200001202231.RAA27334@eric.cnri.reston.va.us> <20000121004112.B987@vet.uu.nl> <14471.41161.944820.970237@bitdiddle.cnri.reston.va.us> Message-ID: <3887A3BC.A02ECDBF@prescod.net> Jeremy Hylton wrote: > > ... > > A related worry is that the type checking will have unexpected effects > on performance when there is an interaction between checked and > unchecked code. Guido showed the potentially painful example of > passing a list containing thousands of elements from unchecked code to > an object declared as "def (int list) -> int"; this seems to imply a > runtime check to verify that all of the elements of the list are ints. > I worry that there would be subtler cases with similar performance > hits. This is a fairly serious issue but my instinct is to say: "if you need optimal performance, turn off runtime type checking" which you'll need to do anyhow. This example is no worse than an O(n) assertion. I can't, off the top of my head, think of a case when this occurs "more subtly" but we will have to watch out for those. -- Paul Prescod - ISOGEN Consulting Engineer speaking for himself Earth will soon support only survivor species -- dandelions, roaches, lizards, thistles, crows, rats. Not to mention 10 billion humans. - Planet of the Weeds, Harper's Magazine, October 1998 From scott@chronis.pobox.com Fri Jan 21 00:43:58 2000 From: scott@chronis.pobox.com (scott) Date: Thu, 20 Jan 2000 19:43:58 -0500 Subject: [Types-sig] Static typing: Towards closure? In-Reply-To: <20000121010224.A1089@vet.uu.nl> References: <20000120231625.A899@vet.uu.nl> <000101bf63a1$849f9a80$6ea0143f@tim> <20000121010224.A1089@vet.uu.nl> Message-ID: <20000120194358.A74972@chronis.pobox.com> Another take on the matter: static typing to me seems benefitial to coding projects that are larger than your average python project. I wouldn't use it much in scripts or single modules, but I would use it and benefit greatly once a project is of a scale that requires thinking about interfaces to begin with. Large projects in python currently have a feel of a never-ending testing phase to me, primarily because of the lack of ability to find those errors at compile time. Regarding the standard library -- I would be much more confident in it if it were staticly typed, but when I browsed library code, I'd like to see it non typed, especially if I were just beginning to look at it and python in general. Despite this, I do believe that as the syntax gets flushed out and after several revisions, static typing will actually increase the readability of code significantly. It's not really there yet, but I do think it will get there. scott On Fri, Jan 21, 2000 at 01:02:24AM +0100, Martijn Faassen wrote: > Tim Peters wrote: > [I say def foo: integer is too much like decl foo: integer] > > Except these don't look like blocks to me at all. > > > > def foo: > > integer > > > > looks like a block. That is, a colon that introduces a block is usually the > > last significant character on a line (always, for people like me). > > I write code like that too, but unfortunately not everybody does... > > [snip Tim (Yoda?) discussing 'the'] > > [static types shouldn't be required; 10% extra lines doesn't necessarily > mean 10% extra effort, type change propagation difficulties causing lots > of reworking of code] > > > premature-static-typing-is-the-root-of-all-evil-ly > > Agreed -- you expressed it better than I could. I don't want to end up > not daring to change anything, because types could break, and I don't want > to end up having to engineer complicated interfaces before I can actually > test them. In C++ (the statically typed language I'm most familiar with) > you can easily end up in just this situation. > > Regards, > > Martijn > > > _______________________________________________ > Types-SIG mailing list > Types-SIG@python.org > http://www.python.org/mailman/listinfo/types-sig From faassen@vet.uu.nl Fri Jan 21 01:00:40 2000 From: faassen@vet.uu.nl (Martijn Faassen) Date: Fri, 21 Jan 2000 02:00:40 +0100 Subject: [Types-sig] Static typing: Towards closure? In-Reply-To: <3887A13E.36E91B2F@prescod.net> References: <200001191722.MAA19286@eric.cnri.reston.va.us> <20000120231625.A899@vet.uu.nl> <200001202231.RAA27334@eric.cnri.reston.va.us> <20000121004112.B987@vet.uu.nl> <3887A13E.36E91B2F@prescod.net> Message-ID: <20000121020040.A1144@vet.uu.nl> Paul Prescod wrote: > Martijn Faassen wrote: > > > .... > > > If type-annotated Python is infecteous (Skip I think called this the 'const' > > propagation problem, like the way you need to const-correct everything in C++ > > if you use const at all), we have a problem. > > We started out with the lack of infectiousness as one of the most > immutable design goal. > > Achieving this is really not hard at all. When you call from Python > (dynamically typed) into C or Java (statically typed) you don't have to > add type declarations to your Python code, do you? Good point. The other way around (from C into Python) however, you need to add checks to your C code to make sure what Python returns is of the type you want. It is the checked module calling an unchecked module that could cause a problem, the other way around is indeed safe enough, as you point out. Because of this, people may be inclined to just add type annotations everywhere. Especially if the compiler rejects many cases of checked calling unchecked, as it surely would. > So why would you need > to add them if you call from dynamically typed Python into statically > typed Python. > > "const" is infectious in C++ because that's the whole point. It was > designed to be infectious. It would have been useless if it were not > infectious. But it causes a problem if you call non-const-correct modules from your const-correct one. You have to go const-correct that module and anything it calls, and so on. I'm making the analogy with checked and unchecked. > > I'm still worried we'll end > > up with a language where there's a type-propagation problem; if we use > > type annotations somewhere, we may have to add them everywhere. My worries > > may not be justified at all; I suppose we'll have to have some practical > > experience with it before we can really know. > > We do have experience and we do know. Use JPython, for example. Hm, you're right. I haven't used JPython. From the docs I couldn't immediately figure out if you can call JPython functions from Java. If so, how does the system ensure that what JPython returns is of the right type? Anyway, my mind is probably not up to this discussion in its present state -- please excuse me any nonsense I'm uttering. No-I'm-not-arguing-just-curious-:)-ly yours, Martijn From scott@chronis.pobox.com Fri Jan 21 01:34:17 2000 From: scott@chronis.pobox.com (scott) Date: Thu, 20 Jan 2000 20:34:17 -0500 Subject: [Types-sig] Static typing: Towards closure? In-Reply-To: <200001201849.NAA21949@eric.cnri.reston.va.us> References: <20000119171832.B63748@chronis.pobox.com> <200001192249.RAA20508@eric.cnri.reston.va.us> <20000119185742.C64311@chronis.pobox.com> <200001201849.NAA21949@eric.cnri.reston.va.us> Message-ID: <20000120203417.A75197@chronis.pobox.com> On Thu, Jan 20, 2000 at 01:49:06PM -0500, Guido van Rossum wrote: > > From: scott [...] > > I currently (ab)use the decl keyword both for defining new types and > for declaring the types of objects; decl T = S defines a new type T to > be a new name for type S (which could be a type expression), while > decl v: T declares a variable v of type T. > > This uses only one keyword and is unambiguous. Can't get better than > that. :-) but you have a second... 'typecase', and the (ab)use of a keyword may also share meaning in that construct or the 'interface' construct, or wherever you have proposed a keyword. The decl keyword is not the only overloadable one. For example, lets say we have two keywords, one for declarations (not 'typedefs' or defining new datatypes), and one for everything else: decl is used for the declaration of a variable, and nothing else. decl x: int decl y: string decl f: def(Int) -> [Int] say the grammar for a decl statement is 'decl' NAME ':' type_expr and type_expr expands to roughly what we've all been using. now, for all kinds of 'typedefs' or "named definitions of collections of type information", you have another keyword, 'datatype'. You can use this in distinct ways to distinguish between datatypes that are a collection of interface info and datatypes that are a collection of type info: a regular named type: datatype Record = (int, string, int, float) an interface (or use the 'interface' keyword here if you can bear a third keyword) datatype Stack: decl push(~T) -> None decl pop() -> ~T datatype StackWithIndex(Stack): decl __getitem__: def(int) -> ~T now, for 'typecase', you could, for example change the grammar for an if_stmt from if_stmt: 'if' test ':' suite ('elif' test ':' suite)* ['else' ':' suite] to if_stmt: 'if' (test | typetest) ':' suite ('elif' (test | typetest) ':' suite)* ['else': ':' suite] typetest: 'datatype' '(' dotted_name ')' 'is' type_expr ('and' 'datatype' '(' dotted_name ')' 'is' type_expr)* yielding two keywords that are arguable meaningful ((ab)used well) and that fit up to 3 keywords from your proposal: 'decl', 'interface', 'typecase'. Additionally, you have a construct that much more closely resembles existing (non type checked) code: if foo is None: foo = default_val_for_foo becomes if datatype(foo) is None: foo = default_val_for_foo rather than typecase(foo): None, x: ... which seems much more convenient, though the semantics are a little trickier because you don't have to cover all the cases (but I don't think that's too hard to type check, especially given all the typing it can save ;) The point being not necessarily that this grammar is better, but that there are other arguably cleaner ways of putting it all together, and we should definitely pursue them to whatever extent we want python to remain a *really* good looking language. scott From scott@chronis.pobox.com Fri Jan 21 02:27:30 2000 From: scott@chronis.pobox.com (scott) Date: Thu, 20 Jan 2000 21:27:30 -0500 Subject: [Types-sig] Static typing: Towards closure? In-Reply-To: <200001201849.NAA21949@eric.cnri.reston.va.us> References: <20000119171832.B63748@chronis.pobox.com> <200001192249.RAA20508@eric.cnri.reston.va.us> <20000119185742.C64311@chronis.pobox.com> <200001201849.NAA21949@eric.cnri.reston.va.us> Message-ID: <20000120212730.B75651@chronis.pobox.com> On Thu, Jan 20, 2000 at 01:49:06PM -0500, Guido van Rossum wrote: > > From: scott > [...] > > > > datatype FIRSTNAME -> FIRSTNAME = string > > datatype LASTNAME -> LASTNAME = string > > datatype NICKNAME -> NICKNAME = string > > > > def get_nickname_from_firstname(n: FIRSTNAME) -> NICKNAME: > > # do stuff > > > > def get_nickname_from_lastname(n: LASTNAME) -> NICKNAME: > > # do stuff > > > > > > # To use this, you need a constructor in order to produce objects > > # with the new datatypes: > > > > decl a: FIRSTNAME > > a = FIRSTNAME("scott") > > print get_nickname_from_lastname(a) # ERROR > > > > By requiring the casts, there is also a good argument that it makes > > code more readable. [.. agreement that this kind of type checking wouldn't be used in the common case for python...] > > What do you call a user defined cast? Just the line FIRSTNAME("scott") in the above example. scott From gstein@lyra.org Fri Jan 21 02:34:19 2000 From: gstein@lyra.org (Greg Stein) Date: Thu, 20 Jan 2000 18:34:19 -0800 (PST) Subject: [Types-sig] Static typing: Towards closure? In-Reply-To: <20000121013812.A1120@vet.uu.nl> Message-ID: On Fri, 21 Jan 2000, Martijn Faassen wrote: > Guido van Rossum wrote: > > Martijn, I won't discuss the implied interfaces idea any further on > > the list. I just find it too absurd. > > Okay, then I won't point out that currently Python allows interfaces that > are even more implicit. :) [whoops, sorry, I'll shut up now] While a class definition *does* imply an interface(*), I think it was your concept of deducing an interface from multiple classes that was the problem. If you have: class Foo implements Alpha: ... class Bar implements Alpha: ... Then Alpha will be constructed from the definitions of Foo and Bar. There is no type-checking that Foo and Bar actually conform to the developer's intention of what Alpha is supposed to mean. Sure, "Alpha" now has *some* meaning, but it might not be the intended meaning. Therefore, it provides no assistance with checking. (*) Note that my type-checker prototype actually builds an interface object based on a class definition. It then uses this interface during the rest of the checking to check whether clients are using the class instances properly. In other words, it is an existence proof of creating and using an implied interface from a class definition. >... > I hadn't considered this distinction between checked and unchecked fully > enough; I'll have to think this one over. Can an unchecked module have > type annotations too, and can this cause run-time TypeError exceptions to > be raised? Perhaps this is in your document and I missed it. I would hope that unchecked modules can include declarations. I don't want to have to convert an *entire* module to be "checked" when I only want type checking in a couple key areas. > Note that runtime checking might help avoid the list of 1000 ints problem > in some cases; this list can be declared to be of ints and this can > be checked at runtime, so that passing it into checked function can > proceed without overhead. But, if this kind of thing is allowed, you > could very well have a type-infection problem; in order to avoid the > overhead people are pushed to add checks everywhere. The runtime checks will always be present on function entry. The function could be called from an unchecked module. Caveat: the runtime check would be eliminated if the function was compiled while the -O switch was present. This can be stated explicitly, or can be deduced if we say function-entry checks use the semantics of the '!' operator (which has explicit semantics with regard to -O). > [snip] > > But I don't think this will affect your ability to write unchecked > > modules at all. The 'const' example doesn't apply here. fyi, this problem is usually called "const poisoning". Note that I plan to submit patches to the Python core at some point to const-poison it :-) I tried a partial version once, but Guido rejected it saying he wanted all or nothing... hehe. Cheers, -g -- Greg Stein, http://www.lyra.org/ From guido@CNRI.Reston.VA.US Fri Jan 21 02:35:49 2000 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Thu, 20 Jan 2000 21:35:49 -0500 Subject: [Types-sig] Static typing: Towards closure? In-Reply-To: Your message of "Thu, 20 Jan 2000 20:34:17 EST." <20000120203417.A75197@chronis.pobox.com> References: <20000119171832.B63748@chronis.pobox.com> <200001192249.RAA20508@eric.cnri.reston.va.us> <20000119185742.C64311@chronis.pobox.com> <200001201849.NAA21949@eric.cnri.reston.va.us> <20000120203417.A75197@chronis.pobox.com> Message-ID: <200001210235.VAA28425@eric.cnri.reston.va.us> Scott suggests a perhaps more Pythonic way to spell a typecase statement: > now, for 'typecase', you could, for example change the grammar for an > if_stmt from > if_stmt: 'if' test ':' suite ('elif' test ':' suite)* ['else' ':' suite] > to > if_stmt: 'if' (test | typetest) ':' suite ('elif' (test | typetest) ':' suite)* ['else': ':' suite] > typetest: 'datatype' '(' dotted_name ')' 'is' type_expr ('and' 'datatype' '(' dotted_name ')' 'is' type_expr)* > > yielding two keywords that are arguable meaningful ((ab)used well) and > that fit up to 3 keywords from your proposal: 'decl', 'interface', > 'typecase'. Additionally, you have a construct that much more closely > resembles existing (non type checked) code: > > if foo is None: > foo = default_val_for_foo > > becomes > > if datatype(foo) is None: > foo = default_val_for_foo > > rather than > > typecase(foo): > None, x: > ... > > which seems much more convenient, though the semantics are a little > trickier because you don't have to cover all the cases (but I don't > think that's too hard to type check, especially given all the typing > it can save ;) Thinking further, I realize that we *already* have a function that does this: isinstance(). The rule could be that if the test in an 'if' or 'elif' statement hasd the form isinstance(, ) where dottedname1 is a statically known variable and dottedname2 is a statically known type name, the type of is considered to be in the corresponding block. This might even let us write things like if isinstance(foo, [long]): ... or if isinstance(bar, int|long|float): ... if we are creative with operator overloading -- although I don't see a lot of use for the latter since the whole point of a typecase is to disambiguate unions... Potential problem: # x is a global decl x: int|float if isinstance(x, int): ...call some function which as a side effect changes x into a float... ...use x as an int... # ERROR, but not detected statically > The point being not necessarily that this grammar is better, but that > there are other arguably cleaner ways of putting it all together, and > we should definitely pursue them to whatever extent we want python to > remain a *really* good looking language. Agreed... --Guido van Rossum (home page: http://www.python.org/~guido/) From gstein@lyra.org Fri Jan 21 02:45:14 2000 From: gstein@lyra.org (Greg Stein) Date: Thu, 20 Jan 2000 18:45:14 -0800 (PST) Subject: [Types-sig] what the heck? (was: Static typing: Towards closure?) In-Reply-To: <20000121004112.B987@vet.uu.nl> Message-ID: On Fri, 21 Jan 2000, Martijn Faassen wrote: > Guido van Rossum wrote: >... > > I don't like this; the colon still looks best to me. > > This may be because the colon is common in other languages for this > purpose (and shorter) -- but I am still worried it may not be good in Python. I'd say consensus is against you here :-). The colon is not ambiguous, is historically used in declarations, is easy to parse visually, and is quick to type. Can't be beat :-) > > > II) The stubbing problem. > > > > > def foo(a, b) -> integer: > > > pass > > > > > > will presumabily cause the typechecker to barf. I'm not quite sure if this > > > is really bad, but I just wanted to bring it to people's attention. > > > > Do this the same say as in Java or C++: return a dummy value. > > Yes, I thought of that, and it may be enough. I just thought to bring the > breakdown of this common usage to people's attention. :) "Breakdown of this common usage" ?? Nobody adds ->integer to their Python code today, so I don't see it as being very common. If people add that, then they will have to deal with the consequences. Namely, that they'll get a compile-time error. Nothing wrong with that. ========= Okay, now we get to the big "HUH?!" in this post... >... > > > As the code solidifies we may want to do more checking, so we > > > can start to explicitize the interfaces. This is similar to the pattern we're > > > hoping to accomplish by adding static typechecking -- it should be possible > > > to code in the dynamic way and then tighten up code by adding type annotations > > > later. > > > > It sounds like a much better thing to do is to leave out all type > > declarations during prototyping, and add them as you start agreeing on > > what the interfaces should be. > > You'd have to leave out all type declarations in the code that *uses* > instances of your classes, then. > > class Foo: > def a(self): > pass > > class Bar: > def a(self): > pass > > def b(self): > pass > > # this doesn't seem to work right > def use1(obj: Foo | Bar): > obj.b() # what happens here? Is this allowed? or is an intersection taken? It fails. Period. You have stated that "obj" might be a Foo, and we know that Foo does not implement the "b" method. What else *should* happen here? I cannot imagine any other reasonable response from the compiler. > > obj = Foo() > use1(obj) > > # this gives a compile time error but we can't pass in any instances of > # Bar. > def use2(obj: Foo): > obj.b() # this causes the error Of course. Again, Foo.b does not exist. It should cause an error. You have also stated that you don't want Bar objects passed. What's the big deal? Why is this behavior not reasonable? > # so we have to do this in order to be able to use both Foo and Bar here: > def use3(obj): > obj.b() # run-time error! Yes, a run-time error. What's your point? > # now, if Foo and Bar both stated they implemented the interface Alpha, > # the compiler could deal with this: > def use4(obj: Alpha): > obj.b() # compile time error We don't need Alpha to do this. The original Foo|Bar did the same. I don't understand your point. The behavior is all very well understood and quite clean. > Isn't this a good intermediate step between having an explicit interface > and not having any interface at all? I have an idea of what the compiler should be doing (as specified above) and think that it works very well. I do *not* understand what you are saying. Yes, there is an implied interface from the definition of Foo, and the invocation "obj.b()" fails when used against that implied interface. BUT: we can entirely omit the word "interface" from this point, and just state that we know "obj.b() will fail because the Foo class does not define a 'b' method." Eliminating the word "interface" from the discussion is Guido's current position. I believe this is entirely possible. At an implementation level, I believe there might be an interface-like object, but that's for another discussion. > [note: it would also be possible to give Foo and Bar a common baseclass and > require that type to be used: > > def use5(obj: Common): > obj.b() # compile time error, not defined by Common Yes. What else is supposed to happen? Really: where are you going with this? I just don't understand. Cheers, -g -- Greg Stein, http://www.lyra.org/ From scott@chronis.pobox.com Fri Jan 21 02:51:36 2000 From: scott@chronis.pobox.com (scott) Date: Thu, 20 Jan 2000 21:51:36 -0500 Subject: [Types-sig] Static typing: Towards closure? In-Reply-To: <200001210235.VAA28425@eric.cnri.reston.va.us> References: <20000119171832.B63748@chronis.pobox.com> <200001192249.RAA20508@eric.cnri.reston.va.us> <20000119185742.C64311@chronis.pobox.com> <200001201849.NAA21949@eric.cnri.reston.va.us> <20000120203417.A75197@chronis.pobox.com> <200001210235.VAA28425@eric.cnri.reston.va.us> Message-ID: <20000120215136.A75907@chronis.pobox.com> On Thu, Jan 20, 2000 at 09:35:49PM -0500, Guido van Rossum wrote: > Scott suggests a perhaps more Pythonic way to spell a typecase > statement: > [...] > > Thinking further, I realize that we *already* have a function that > does this: isinstance(). The rule could be that if the test in an > 'if' or 'elif' statement hasd the form isinstance(, > ) where dottedname1 is a statically known variable and > dottedname2 is a statically known type name, the type of > is considered to be in the corresponding block. > > This might even let us write things like > > if isinstance(foo, [long]): ... > > or > > if isinstance(bar, int|long|float): ... > > if we are creative with operator overloading -- although I don't see a > lot of use for the latter since the whole point of a typecase is to > disambiguate unions... > > Potential problem: > > # x is a global > decl x: int|float > > if isinstance(x, int): > ...call some function which as a side effect changes x into a float... > ...use x as an int... # ERROR, but not detected statically Do you mean to add the word 'isinstance' as a keyword in the grammar, or to use the builtin function? As a keyword, it seems very tricky to make it call some internal 'isinstance' that is no longer available as a builtin. As a builtin, we have lots of problems: if sys.argv[1:] or isinstance(x, int): if isinstance(x, int) == 1: etc. hmm. scott From faassen@vet.uu.nl Fri Jan 21 03:06:36 2000 From: faassen@vet.uu.nl (Martijn Faassen) Date: Fri, 21 Jan 2000 04:06:36 +0100 Subject: [Types-sig] Re: what the heck? (was: Static typing: Towards closure?) In-Reply-To: References: <20000121004112.B987@vet.uu.nl> Message-ID: <20000121040636.A1300@vet.uu.nl> Greg Stein wrote: [snip] > Okay, now we get to the big "HUH?!" in this post... [snip] > > You'd have to leave out all type declarations in the code that *uses* > > instances of your classes, then. > > > > class Foo: > > def a(self): > > pass > > > > class Bar: > > def a(self): > > pass > > > > def b(self): > > pass > > > > # this doesn't seem to work right > > def use1(obj: Foo | Bar): > > obj.b() # what happens here? Is this allowed? or is an intersection taken? > > It fails. Period. You have stated that "obj" might be a Foo, and we know > that Foo does not implement the "b" method. This would mean that an intersection of the interface of class Foo and Bar is taken, and if you call any method that's not on either, there is a compile time failure. In effect this is a different way to spell implements, except that the '|' makes it possible to do it ad-hoc. This means that something like typedef Alpha = Foo | Bar is creating the same thing as the 'implements' proposal? Or am I understanding this completely wrong? > What else *should* happen here? I cannot imagine any other reasonable > response from the compiler. Me neither, really. :) [snip] > What's the big deal? Why is this behavior not reasonable? No big deal; the behavior is perfectly reasonable, I never claimed it wasn't. > > # so we have to do this in order to be able to use both Foo and Bar here: > > def use3(obj): > > obj.b() # run-time error! > > Yes, a run-time error. What's your point? We wanted to have the advantages of static-type checking, so I was wondering if some intermediate system was possible where you can have some of the features of static type checking without having the full-blown system. This in order to apply some static type checking to Python code that's altered only minimally from its current form. > > # now, if Foo and Bar both stated they implemented the interface Alpha, > > # the compiler could deal with this: > > def use4(obj: Alpha): > > obj.b() # compile time error > > We don't need Alpha to do this. The original Foo|Bar did the same. You're probably right and I'm just looking at it in an upside-down way. If you can create implied interfaces (intersections) using something like: typedef Alpha = Foo | Bar then you have essentially the same problems as the other people who responded to my mails listed. How do you get around those problems? Or is it just intended not to be used often enough to be a problem? > I don't understand your point. The behavior is all very well understood > and quite clean. I must be misunderstanding something if my above speculation is the case (it may very well not be) -- if it is, I'm not quite sure how their valid objections to my different spelling of the same thing make sense.. > > Isn't this a good intermediate step between having an explicit interface > > and not having any interface at all? > > I have an idea of what the compiler should be doing (as specified above) > and think that it works very well. I do *not* understand what you are > saying. I understood that. I must've been expressing myself really badly tonight. > Yes, there is an implied interface from the definition of Foo, and the > invocation "obj.b()" fails when used against that implied interface. > > BUT: we can entirely omit the word "interface" from this point, and just > state that we know "obj.b() will fail because the Foo class does not > define a 'b' method." > > Eliminating the word "interface" from the discussion is Guido's current > position. I believe this is entirely possible. At an implementation level, > I believe there might be an interface-like object, but that's for another > discussion. If you include typedefs that interface-like object is named. Or is this a typedef that is not allowable? > > [note: it would also be possible to give Foo and Bar a common baseclass and > > require that type to be used: > > > > def use5(obj: Common): > > obj.b() # compile time error, not defined by Common > > Yes. What else is supposed to happen? It's completely valid; I wasn't saying anything was _wrong_ with this. > Really: where are you going with this? I just don't understand. I was coming to this from the point of view of a (out of his mind :) user. When writing Python code currently, the dynamic typing allows you to program without a lot of the rigidity that a static typing system brings you. This rigidity can be good for OPT and ERR, but can be bad for PRO, rapid prototyping/speed of development. One tends to get side-tracked into issues of which type to use where instead of actually solving the problem at hand, bad for PRO. Still, static typing can be nice as well. What I was grasping for was some way to use the typechecking system while at the same time being relatively little bothered by the rigidity. So, a way to have _some_ ERR (and DOC), while keeping a lot of PRO. Now, I'm reaching the conclusion that if I understand the semantics of '|' right then that the thing I was talking about already exists. It may also be possible that my head is too fuzzy to make sense of things anymore. :) Did that explain my wacky advocacy at least a little, Greg? Regards, Martijn From gstein@lyra.org Fri Jan 21 03:32:27 2000 From: gstein@lyra.org (Greg Stein) Date: Thu, 20 Jan 2000 19:32:27 -0800 (PST) Subject: [Types-sig] Re: what the heck? (was: Static typing: Towards closure?) In-Reply-To: <20000121040636.A1300@vet.uu.nl> Message-ID: On Fri, 21 Jan 2000, Martijn Faassen wrote: > Greg Stein wrote: >... > > > # this doesn't seem to work right > > > def use1(obj: Foo | Bar): > > > obj.b() # what happens here? Is this allowed? or is an intersection taken? > > > > It fails. Period. You have stated that "obj" might be a Foo, and we know > > that Foo does not implement the "b" method. > > This would mean that an intersection of the interface of class Foo and > Bar is taken, and if you call any method that's not on either, there > is a compile time failure. > > In effect this is a different way to spell implements, except that the > '|' makes it possible to do it ad-hoc. > > This means that something like > > typedef Alpha = Foo | Bar > > is creating the same thing as the 'implements' proposal? Nope. > Or am I understanding this completely wrong? In your original statement, Alpha is the intersection of the Foo and Bar interfaces. The above typedef is very different. Specifically: def f(x: Foo|Bar): ... def g(x: Alpha): ... In the former case, f() might use features of Foo or Bar after determining which you passed. In the latter, it is only using Alpha features. *Very* different semantics. >... > > What else *should* happen here? I cannot imagine any other reasonable > > response from the compiler. > > Me neither, really. :) And this is what gets me. Why the heck bring it up? Just to give us something to read? To make our brains spin some cycles to figure what you were trying to say, when you really weren't? Sorry. I didn't get much sleep last night, so I'm probably being a bit harsh. But I do tend to dislike having to pore through long emails looking for a point, only to find nothing. > [snip] > > What's the big deal? Why is this behavior not reasonable? > > No big deal; the behavior is perfectly reasonable, I never claimed > it wasn't. See point above. >... > We wanted to have the advantages of static-type checking, so I was wondering > if some intermediate system was possible where you can have some of the > features of static type checking without having the full-blown system. This > in order to apply some static type checking to Python code that's altered > only minimally from its current form. I think the current proposal, embodied in Guido's paper and slide deck sufficiently answer the "intermediate" case. > > > # now, if Foo and Bar both stated they implemented the interface Alpha, > > > # the compiler could deal with this: > > > def use4(obj: Alpha): > > > obj.b() # compile time error > > > > We don't need Alpha to do this. The original Foo|Bar did the same. > > You're probably right and I'm just looking at it in an upside-down way. > If you can create implied interfaces (intersections) using something like: > > typedef Alpha = Foo | Bar This is an implied interface, but different from the output of your original proposal. I think that I wasn't clear in my response above. "You don't need Alpha [to generate the error on obj.b]". In other words, you need need the implied construction of Alpha to solve the obj.b problem. >... > > Yes, there is an implied interface from the definition of Foo, and the > > invocation "obj.b()" fails when used against that implied interface. > > > > BUT: we can entirely omit the word "interface" from this point, and just > > state that we know "obj.b() will fail because the Foo class does not > > define a 'b' method." > > > > Eliminating the word "interface" from the discussion is Guido's current > > position. I believe this is entirely possible. At an implementation level, > > I believe there might be an interface-like object, but that's for another > > discussion. > > If you include typedefs that interface-like object is named. Or is this a > typedef that is not allowable? The class object serves the same purpose. There is no need to expose the interface object. Specifically: class Foo: ... a = f() ! Foo In the above example, Foo is a name pointing to a class object. The type-assert uses the class object, rather than its implied interface to perform the type-check. So: we don't need the word "interface". > > > [note: it would also be possible to give Foo and Bar a common baseclass and > > > require that type to be used: > > > > > > def use5(obj: Common): > > > obj.b() # compile time error, not defined by Common > > > > Yes. What else is supposed to happen? > > It's completely valid; I wasn't saying anything was _wrong_ with this. See point further above. > > Really: where are you going with this? I just don't understand. > > I was coming to this from the point of view of a (out of his mind :) user. And coming from a short sleep period, I was getting agitated :-) > When writing Python code currently, the dynamic typing allows you to > program without a lot of the rigidity that a static typing system brings you. > This rigidity can be good for OPT and ERR, but can be bad for PRO, rapid > prototyping/speed of development. One tends to get side-tracked into issues > of which type to use where instead of actually solving the problem at hand, > bad for PRO. Sure. So don't use them. As Guido has pointed out, you will retain the exact same, rapid development capabilities as before. The compiler might issue a few errors that it didn't before, but that is simply because you *did* make an error. You can continue to write huge bodies of code without a single declaration. Personally, I will primarily use them in function definitions (such as "def foo(x:int):"), so that I can get asserts on function-entry. > Still, static typing can be nice as well. What I was grasping for was some > way to use the typechecking system while at the same time being relatively > little bothered by the rigidity. So, a way to have _some_ ERR (and DOC), > while keeping a lot of PRO. Just use less. There is no "poisoning" that will require you to use declarations everywhere. Use it in key places where it helps. Avoid it in others. Cheers, -g -- Greg Stein, http://www.lyra.org/ From tony@metanet.com Fri Jan 21 04:24:14 2000 From: tony@metanet.com (Tony Lownds) Date: Thu, 20 Jan 2000 20:24:14 -0800 (PST) Subject: [Types-sig] Static typing: Towards closure? In-Reply-To: <20000120215136.A75907@chronis.pobox.com> Message-ID: isinstance is a builtin today decl def isinstance(any, class|type) -> 0|1 Sorry to jump into the thread, but I'm curious as to whether this declaration is how it would be written... -Tony On Thu, 20 Jan 2000, scott wrote: > Do you mean to add the word 'isinstance' as a keyword in the grammar, > or to use the builtin function? As a keyword, it seems very tricky to > make it call some internal 'isinstance' that is no longer available as > a builtin. As a builtin, we have lots of problems: > > if sys.argv[1:] or isinstance(x, int): > > if isinstance(x, int) == 1: > > etc. > > hmm. > > scott > > > > _______________________________________________ > Types-SIG mailing list > Types-SIG@python.org > http://www.python.org/mailman/listinfo/types-sig > From Tony Lownds Fri Jan 21 04:46:37 2000 From: Tony Lownds (Tony Lownds) Date: Thu, 20 Jan 2000 20:46:37 -0800 (PST) Subject: [Types-sig] lambdas, classes, tuples, default arguments, map Message-ID: Hi, Here is my 2 cents on the proposal, which looks really good to me overall. 1. Requiring a return type on typed lambdas could help disambiguate things: decl f_pair: def((any,any)) -> any decl f_2: def(any,any) -> any f_2 = lambda x,y: ... # ok f_pair = lambda (x,y): ... # ok f_2 = lambda (x,y)->any: ... # ok f_pair = lambda ((x,y))->any: ... #ok 2. The way I read the proposal, class definitions themselves are valid if each assignment made in the class suite is valid wrt its bases. You can use a class as a type in a decl. Type checks involving such a decl only look at inheritence, not structure. I'm guessing there would be some kind of fundamental base class for the special python methods - __len__ etc. __init__ would have to be treated a bit specially, because subclasses' constructors are usually incompatible with superclass' constructors. Is this right? And can one specify that a subclass' constructor should indeed be compatible with a superclass' init? 3. On the How to spell a tuple issue - the proposal mentions (str*), how about (*str)... decl def f(*args:(*str)) -> int 4. How do you specify default values for parameters in decls? The simple answer is to use "= value" decl def foo(a: int, b: int|None = None) -> int But for cases where the default is a complicated object thats going to be a problem. 5. map has a function signature that is pretty hard to declare, even with parameterization of functions. I hope it will be special-cased rather than under-specified. -Tony From Richard.Colley@alcatel.com.au Fri Jan 21 07:16:52 2000 From: Richard.Colley@alcatel.com.au (Richard Colley) Date: Fri, 21 Jan 2000 18:16:52 +1100 Subject: [Types-sig] Static typing: Towards closure? References: <000101bf63a1$849f9a80$6ea0143f@tim> Message-ID: <00Jan21.181652est.115220@border.alcanet.com.au> See comments in-line... ----- Original Message ----- From: "Tim Peters" To: Sent: Friday, 21 January 2000 10:54 Subject: RE: [Types-sig] Static typing: Towards closure? > Where static typing costs in development time way out of proportion to its > line count is under modification, where "all of a sudden" a conceptually > simple change of type in one place ends up getting manually propagated all > over the place (method X used to return Y but now returns Z, so potentially > all call sites need to get redeclared, as well as potentially all code > that-- directly or indirectly -- "consumes" the return value(s)). What is a > one-line change in untyped Python may require an unbounded number of changes > in typed Python. That one-line change will however cause run-time grief. If you are returning a different type of object without modifying the expectations of the code that "consumes" the return value(s) then how can you expect a correct program. What I'm trying to say, it's not going to be a one-line change in untyped Python. In fact the statically typed Python has the advantage of automatically identifying all candidates for change - the untyped one requires you to guess or wait until runtime. Richard From gstein@lyra.org Fri Jan 21 11:08:19 2000 From: gstein@lyra.org (Greg Stein) Date: Fri, 21 Jan 2000 03:08:19 -0800 (PST) Subject: [Types-sig] namespaces (was: Changing existing class instances) In-Reply-To: <001801bf63f3$414e0c60$ec2d153f@tim> Message-ID: On Fri, 21 Jan 2000, Tim Peters wrote: > [Greg Stein] > > ... > > In other words, I definitely would support a new class > > object behavior that allows us to update a class' set of > > bases and dictionary on the fly. This could then be used > > to support my solution for the recursive type scenario (which, > > in turn, means that we don't have to introduce Yet Another > > Namespace into Python to hold type names). > > Parenthetically, I never grasped the appeal of the parenthetical comment. > Yet Another Namespace for Yet Another Entirely New Purpose seems highly > *desirable* to me! Trying to overload the current namespace set makes it so > much harder to see that these are compile-time gimmicks, and users need to > be acutely aware of that if they're to use it effectively. Note that I > understand (& wholly agree with) the need for runtime introspection. And that is the crux of the issue: I think the names that are assigned to these classes, interfaces, typedefs, or whatever, can follow the standard Python semantics and be plopped into the appropriate namespace. There is no overloading. The compile-time behavior certainly understands what names have what types; in this case, if a name is a "typedecl", then it can remember the *value*, too. When the name is used later, it knows the corresponding value to use. For instance: IntOrString = typedef int|str def foo(x: IntOrString): ... In this example, the type-checker knows that IntOrString is a typedecl. It also knows the *value* of "int|str" so the name IntOrString now has two items associated with it at type-check time: # not "real" syntax, but you get the idea... namespace["IntOrString"] = (TypeDeclarator, int|str) With the above information in hand, the type-checker knows what IntOrString means in the declaration for foo(). The cool benefit is that the runtime semantics are exactly as you would expect: a typedecl object is created and assigned to IntOrString. That object is also associated with the "x" argument in the function object referred to by the name "foo". There is no "overloading" of namespaces. We are using Python namespaces just like they should be, and the type-checker doesn't even have to be all the smart to track this stuff. To get back to the recursive class problem, consider the following code: decl incomplete class Foo decl incomplete class Bar class Foo: decl a: Bar class Bar: decl b: Foo The "decl" statements would create an empty class object and store that into the "current" namespace. There is no need to shove that off into another namespace. When the "class Foo" comes along, the class object is updated with the class definition for Foo. It is conceivable to remove the need for "decl" if you allow "class" and "def" to omit the ": suite" portion of their grammar: class Foo class Bar class Foo: decl a: Bar ... def some_function(x: some_type, y: another_type) -> third_type ... lots o' code ... def some_function(x, y): ... Guido suggested that it may be possible to omit "decl" altogether. Certainly, it can work for member declarations such as: class Foo: a: Bar Anyhow... my point is that a new namespace is not needed. Assuming we want objects for reflection at runtime, then the above proposal states *how* those objects are realized at runtime. Further, the type-checker can easily follow that information and perform the appropriate compile-time checks. No New Namespaces! (lather, rinse, repeat) Cheers, -g -- Greg Stein, http://www.lyra.org/ From tim_one@email.msn.com Fri Jan 21 11:06:53 2000 From: tim_one@email.msn.com (Tim Peters) Date: Fri, 21 Jan 2000 06:06:53 -0500 Subject: [Types-sig] Static typing: Towards closure? In-Reply-To: <00Jan21.181652est.115220@border.alcanet.com.au> Message-ID: <001f01bf63ff$a0b092c0$ec2d153f@tim> [Tim] > What is a one-line change in untyped Python may require > an unbounded number of changes in typed Python. [Richard Colley] > That one-line change will however cause run-time grief. If you > returning a different type of object without modifying the > expctations of the code that "consumes" the return value(s) then > how can you expect a correct program. In Python, very easily -- routinely, even. The problem is that *people* tend to name a most-specific type instead of the most-general type. It's my belief that people (including me!) explicitly declare the types of their functions in languages like Haskell precisely because the language's automatic inference of the most-general type leads to incomprehensible error msgs (I never gave a thought to the most-general type, and don't recognize what the heck it's whining about when it spits one back at me). For a simple example, dicts and lists in Python can both be indexed by integers, with the same thing[i] syntax, and I've very often changed a dict to a list (or vice versa) without any code breaking as a result (a "sparse" problem turns into a "dense" one, or vice versa). But I'm rarely going to think ahead clearly enough to *declare* names as "list | dict". Indeed, for OPT reasons I would be loathe to do so even if I were thinking ahead. So I'll delaying adding type info until I'm sure I've got the right choice -- and sometimes regret it later. Running around changing decls during initial development is simply a waste of time. And this happens "all the time"! I still recall one of my first Python Snorts of Joy, after I had written a GCD algorithm for integers and realized that the exact same code would also compute the GCD of polynomials. Nobody is going to see ahead clearly enough to declare such a function from the start as having arguments and return value of type EuclidianDomain <0.1 wink>. As a more common example, consider Python's ubiquitous but largely undocumented use of the folklore "file-like object" protocol. Many functions written to work "on files" are actually much more broadly applicable in Python, but few people are going to realize that in advance; newbies have no chance of realizing it. This goes on & on. Python wasn't designed with a static view of types in mind, and it's going to be a real strain to shove some common Python practices into a static type system. > What I'm trying to say, it's not going to be a one-line change > in untyped Python. In fact the statically typed Python has > the advantage of automatically identifying all candidates > for change - the untyped one requires you to guess or wait > until runtime. Except that, so far as correctness goes, "guessing or waiting" is often a good bet today, and can approach certainty with increasing Python experience. I'll certainly use static typing for OPT, ERR and DOC *when the code needs one of those* -- but I'm under no illusion that it's a pure win. python-did-remarkably-well-without-any-of-this-the-first- decade-of-its-life-ly y'rs - tim From gstein@lyra.org Fri Jan 21 12:05:34 2000 From: gstein@lyra.org (Greg Stein) Date: Fri, 21 Jan 2000 04:05:34 -0800 (PST) Subject: [Types-sig] namespaces (was: Changing existing class instances) Message-ID: Scott did it again.... :-) Forwarding to types-sig ---------- Forwarded message ---------- Date: Fri, 21 Jan 2000 06:29:09 -0500 From: scott To: Greg Stein Subject: Re: [Types-sig] namespaces (was: Changing existing class instances) On Fri, Jan 21, 2000 at 03:08:19AM -0800, Greg Stein wrote: > On Fri, 21 Jan 2000, Tim Peters wrote: > > [Greg Stein] > No New Namespaces! (lather, rinse, repeat) I agree wholeheartedly for named types. They just fit into existing namesapaces, and are 'first class' objects like everything else in python. Where this becomes difficult is for un-named types. The runtime evironment needs to know the type of an object in order to perform certain checks, like your ! op or typecase or if datatype(x)... however you spell it. But a new namespace seems to be an unclean place to store that info. Where does that info go? An extra PyObject * pointer inside of PyObject *'s? I don't know. That info can't really fit into a name space because there's no name associated with it per se. It's the type of a value. scott From gstein@lyra.org Fri Jan 21 12:37:56 2000 From: gstein@lyra.org (Greg Stein) Date: Fri, 21 Jan 2000 04:37:56 -0800 (PST) Subject: [Types-sig] namespaces In-Reply-To: Message-ID: On Fri, 21 Jan 2000, scott wrote: > On Fri, Jan 21, 2000 at 03:08:19AM -0800, Greg Stein wrote: > > On Fri, 21 Jan 2000, Tim Peters wrote: > > > [Greg Stein] > > No New Namespaces! (lather, rinse, repeat) > > I agree wholeheartedly for named types. They just fit into existing > namesapaces, and are 'first class' objects like everything else in > python. > > Where this becomes difficult is for un-named types. The runtime > evironment needs to know the type of an object in order to perform > certain checks, like your ! op or typecase or if datatype(x)... > however you spell it. > > But a new namespace seems to be an unclean place to store that info. > Where does that info go? An extra PyObject * pointer inside of > PyObject *'s? I don't know. That info can't really fit into a name > space because there's no name associated with it per se. It's the > type of a value. They don't go into namespaces. They are simply evaluated -- just like any other expressions. a = b + c The value "b + c" doesn't go into a namespace. It just resides on the execution stack. Similarly: a = b ! int|str type(int) == type(str) == int|str creates a typedecl object with two alternatives (the int and str objects). This typedecl object is then used as an operand for '!'. In typedecl.py in my prototype, there is a class named TDAlternates (a derived class of TypeDeclarator). The code above could be expanded like so: int = type(1) str = type('') __stack_1 = typedecl.TDAlternates(int, str) __stack_2 = b if __debug__ and not isinstance(__stack_2, __stack_1): raise TypeAssertionError a = __stack_2 In this example, __stack_N represents a value on the execution stack. It doesn't truly reside in a namespace. Assume that appropriate stack manipulation (e.g. "pop" or "dup") is in there, where needed. Also, presume that isinstance() understands how to deal with TypeDeclarator instances. In my prototype, isinstance would do something like: __stack_1.check(__stack_2). Basically, isinstance can take any of the following for a second argument: - a PyTypeType object (e.g. type(1)) - a PyClassObject object - a TypeDeclarator instance The first two are allowed today, the third would be added. Or whatever the equivalent is in the final implementation. The fourth is reserved for the time when interfaces are added (although, in my prototype, an interface is just a subclass of TypeDeclarator). For example: def isinstance(value, type_value): t = type(type_value) if t is types.TypeType: return type(value) is type_value if t is types.ClassType: if type(value) is not types.InstanceType: return 0 cls = value.__class__ return issubclass(cls, type_value) assert type(type_value) is types.InstanceType assert issubclass(type_value.__class__, TypeDeclarator) return type_value.check(value) Note that the TypeDeclarator classes in the prototype have a little more complex notion of type(value). You don't want to look at an InstanceType, but the concept "this is an instance of Foo." Given a value, typedecl.build_typedecl(value) will return an appropriate TypeDeclarator instance. Last year, I proposed expanding isinstance() to take a callable. I hereby retract that :-) I think using .check() is Better. Making a TypeDeclarator callable could be handy: IntStack = typedef Stack s = IntStack() We want to be able to call IntStack to construct an instance, but we also want to call the thing during isinstance() processing. Cheers, -g -- Greg Stein, http://www.lyra.org/ From scott@chronis.pobox.com Fri Jan 21 13:36:48 2000 From: scott@chronis.pobox.com (scott) Date: Fri, 21 Jan 2000 08:36:48 -0500 Subject: [Types-sig] namespaces In-Reply-To: References: Message-ID: <20000121083648.A78943@chronis.pobox.com> Stack evaluation looks good, that clears up a lot for me re: runtime behavior. but there are problems with both ! (pushes type checking errors from compile time to runtime) and isinstance: it's grammatically ambiguous as a designator of sections of code blocks which may assume disambiguated alternate types. This becomes quite evident if you try to write a checker with a function call that is the runtime disambiguating agent. Are there any other ideas or workarounds with these ideas that address these problems? scott On Fri, Jan 21, 2000 at 04:37:56AM -0800, Greg Stein wrote: > On Fri, 21 Jan 2000, scott wrote: > > On Fri, Jan 21, 2000 at 03:08:19AM -0800, Greg Stein wrote: > > > On Fri, 21 Jan 2000, Tim Peters wrote: > > > > [Greg Stein] > > > No New Namespaces! (lather, rinse, repeat) > > > > I agree wholeheartedly for named types. They just fit into existing > > namesapaces, and are 'first class' objects like everything else in > > python. > > > > Where this becomes difficult is for un-named types. The runtime > > evironment needs to know the type of an object in order to perform > > certain checks, like your ! op or typecase or if datatype(x)... > > however you spell it. > > > > But a new namespace seems to be an unclean place to store that info. > > Where does that info go? An extra PyObject * pointer inside of > > PyObject *'s? I don't know. That info can't really fit into a name > > space because there's no name associated with it per se. It's the > > type of a value. > > They don't go into namespaces. They are simply evaluated -- just like any > other expressions. > > a = b + c > > The value "b + c" doesn't go into a namespace. It just resides on the > execution stack. Similarly: > > a = b ! int|str > > type(int) == > type(str) == > > int|str creates a typedecl object with two alternatives (the int and str > objects). This typedecl object is then used as an operand for '!'. > > In typedecl.py in my prototype, there is a class named TDAlternates (a > derived class of TypeDeclarator). The code above could be expanded like > so: > > int = type(1) > str = type('') > __stack_1 = typedecl.TDAlternates(int, str) > __stack_2 = b > if __debug__ and not isinstance(__stack_2, __stack_1): > raise TypeAssertionError > a = __stack_2 > > In this example, __stack_N represents a value on the execution stack. It > doesn't truly reside in a namespace. Assume that appropriate stack > manipulation (e.g. "pop" or "dup") is in there, where needed. Also, > presume that isinstance() understands how to deal with TypeDeclarator > instances. In my prototype, isinstance would do something like: > __stack_1.check(__stack_2). > > Basically, isinstance can take any of the following for a second argument: > > - a PyTypeType object (e.g. type(1)) > - a PyClassObject object > - a TypeDeclarator instance > > The first two are allowed today, the third would be added. Or whatever the > equivalent is in the final implementation. The fourth is reserved for the > time when interfaces are added (although, in my prototype, an interface is > just a subclass of TypeDeclarator). For example: > > def isinstance(value, type_value): > t = type(type_value) > if t is types.TypeType: > return type(value) is type_value > if t is types.ClassType: > if type(value) is not types.InstanceType: > return 0 > cls = value.__class__ > return issubclass(cls, type_value) > assert type(type_value) is types.InstanceType > assert issubclass(type_value.__class__, TypeDeclarator) > return type_value.check(value) > > Note that the TypeDeclarator classes in the prototype have a little more > complex notion of type(value). You don't want to look at an InstanceType, > but the concept "this is an instance of Foo." Given a value, > typedecl.build_typedecl(value) will return an appropriate TypeDeclarator > instance. > > Last year, I proposed expanding isinstance() to take a callable. I hereby > retract that :-) I think using .check() is Better. Making a TypeDeclarator > callable could be handy: > > IntStack = typedef Stack > s = IntStack() > > We want to be able to call IntStack to construct an instance, but we also > want to call the thing during isinstance() processing. > > Cheers, > -g > > -- > Greg Stein, http://www.lyra.org/ > > > _______________________________________________ > Types-SIG mailing list > Types-SIG@python.org > http://www.python.org/mailman/listinfo/types-sig From guido@CNRI.Reston.VA.US Fri Jan 21 13:43:55 2000 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Fri, 21 Jan 2000 08:43:55 -0500 Subject: [Types-sig] namespaces In-Reply-To: Your message of "Fri, 21 Jan 2000 08:36:48 EST." <20000121083648.A78943@chronis.pobox.com> References: <20000121083648.A78943@chronis.pobox.com> Message-ID: <200001211343.IAA28893@eric.cnri.reston.va.us> [scott] > Stack evaluation looks good, that clears up a lot for me re: runtime > behavior. but there are problems with both ! (pushes type checking > errors from compile time to runtime) and isinstance: it's > grammatically ambiguous as a designator of sections of code blocks > which may assume disambiguated alternate types. This becomes quite > evident if you try to write a checker with a function call that is the > runtime disambiguating agent. Scott, I don't understand the last two sentences. Can you give an example? (Have you written a compiler before? It seems quite possible to me to define either one unambiguously.) > Are there any other ideas or workarounds with these ideas that > address these problems? --Guido van Rossum (home page: http://www.python.org/~guido/) From gstein@lyra.org Fri Jan 21 14:19:05 2000 From: gstein@lyra.org (Greg Stein) Date: Fri, 21 Jan 2000 06:19:05 -0800 (PST) Subject: [Types-sig] namespaces In-Reply-To: <20000121083648.A78943@chronis.pobox.com> Message-ID: On Fri, 21 Jan 2000, scott wrote: > Stack evaluation looks good, that clears up a lot for me re: runtime > behavior. but there are problems with both ! (pushes type checking > errors from compile time to runtime) '!' can *definitely* invoke a compile-time check: x = 5 y = x!string # Boom! The operator also provides information to the type-checker. > and isinstance: it's > grammatically ambiguous as a designator of sections of code blocks > which may assume disambiguated alternate types. This becomes quite > evident if you try to write a checker with a function call that is the > runtime disambiguating agent. I agree. I posed to Guido a while back that I do not believe we can use isinstance() to propertly typecheck the following code: if isinstance(x, int): return x + 5 if isinstance(x, str): return x + 'hi' Specifically, because the two parameters are arbitrary expressions, it will be reasonably complicated to understand the type specified by the second argument, and to determine that the first argument is an lvalue (meaning we can record the type on the lvalue). Moreover, it is difficult to "temporarily" record "int" for "x" within the if: block (up to that point, we think x is int|str). Strictly: yes, we could typecheck it (IFF the user followed predefined patterns and restrictions). Would I want to write the code? Gack. No :-) I'd rather avoid those restrictions and rely on a clean mechanism. (described below) > Are there any other ideas or > workarounds with these ideas that address these problems? I am advocating against a new typecase statement. Specifically, Python does not have a switch statement right now for reasons "X" and "Y" (exercise for the reader). I think those same reasons could be applied to a typecase statement. Here is Guido's example, and my interpretation: decl x: int | string | None | [any] | ... typecase x: int, i: print "an int: %d" % i string, s: print "a string: %s" % `s` None: print "nothing" else: print "don't know" if isinstance(x, int): print "an int: %d" % (x!int) elif isinstance(x, string): print "a string: %s" % `x!string` elif isinstance(x, None): print "nothing" else: print "don't know" Of course, that is a *very* literal translation. We only need the (x!int) form if the type-checker compares formats against input types. The x!string is unnecessary since `` works on any input type. Using isinstance to check for None is too heavyweight. But, let's use a "better" example: def munge(x: int|str)->int|str: # the typecase form: typecase x: int, i: return i + 5 # can't use else: here, because we need the "casting" performed by # the typecase statement to assign a value to "s" str, s: return s + 'hi' # the Greg form: if isinstance(x, int): return x!int + 5 # this could be an else: or omitted (because of the preceding return) if isinstance(x, str): return x!str + 'hi' Specifically, I think we should use if/elif/else statements rather than a typecase. In combination with the '!' operator, we successfully pass the type-check step without errors. It is true that isinstance() is a bit more wordy, but I doubt the above code patterns will be all that common. (can somebody supply some examples?) Here is what I find will be *very* common: decl checked_module import third_party_unchecked_module as tpum x = tpum.func() # by definition, return type is "any" func_taking_str(x!str) # we know it is a string and need to say so In other words, I think the '!' will be common when dealing with unchecked modules. I don't think it will be used all that often to narrow types down from a set of alternatives (which is the purpose of a typecase statement). Therefore, I'd rather avoid introducing the typecase syntax -- not enough benefit for the cost of the additional syntax. Cheers, -g -- Greg Stein, http://www.lyra.org/ From gstein@lyra.org Fri Jan 21 14:35:19 2000 From: gstein@lyra.org (Greg Stein) Date: Fri, 21 Jan 2000 06:35:19 -0800 (PST) Subject: [Types-sig] Static typing: Towards closure? In-Reply-To: <200001210235.VAA28425@eric.cnri.reston.va.us> Message-ID: On Thu, 20 Jan 2000, Guido van Rossum wrote: >... > Thinking further, I realize that we *already* have a function that > does this: isinstance(). The rule could be that if the test in an > 'if' or 'elif' statement hasd the form isinstance(, > ) where dottedname1 is a statically known variable and > dottedname2 is a statically known type name, the type of > is considered to be in the corresponding block. Per my previous mail, I'm advocating against basing a typecase mechanism on something that requires the user to fit into patterns and restrictions like this. In other words, I hope that I don't have to remember to only use dotted names in an isinstance() to receive compile-time benefits. What happens if I forget? Error? Nothing? Warning? Most of Python is extremely intuitive and obvious. This strikes me as the kind of "magic" that we'd like to avoid. >... > # x is a global > decl x: int|float > > if isinstance(x, int): > ...call some function which as a side effect changes x into a float... > ...use x as an int... # ERROR, but not detected statically IMO, if we just rely on the programmer to perform the typecase (at runtime!) with isinstance() explicitly and to use the '!' operator to handle the compile-time narrowing, then we are set. The programmer should be aware of side effects. If not, the type-assert operator will catch it (and tell him the code is too convoluted :-) This kind of boils down to how conservative do we get with globals? decl a: int|str decl b: int a = 1 foo() # what is the type of a? b = a # should fail b = a!int # no problem # a is an int again b = a "projecting" a type forward from an isinstance() (as you demo above) or within a typecase suite would have similar problems. typecase a: int: # type-checker labels "a" as an "int" b = a # passes foo() # what is the type of a? isinstance and '!' seem to be much more clear and robust. And a reminder for ourselves :-) ... we should be careful about labeling a global (at all and/or across a func call). Cheers, -g -- Greg Stein, http://www.lyra.org/ From gstein@lyra.org Fri Jan 21 14:43:47 2000 From: gstein@lyra.org (Greg Stein) Date: Fri, 21 Jan 2000 06:43:47 -0800 (PST) Subject: [Types-sig] performance (was: Static typing: Towards closure?) In-Reply-To: <3887A3BC.A02ECDBF@prescod.net> Message-ID: On Thu, 20 Jan 2000, Paul Prescod wrote: > Jeremy Hylton wrote: > > ... > > A related worry is that the type checking will have unexpected effects > > on performance when there is an interaction between checked and > > unchecked code. Guido showed the potentially painful example of > > passing a list containing thousands of elements from unchecked code to > > an object declared as "def (int list) -> int"; this seems to imply a > > runtime check to verify that all of the elements of the list are ints. > > I worry that there would be subtler cases with similar performance > > hits. > > This is a fairly serious issue but my instinct is to say: "if you need > optimal performance, turn off runtime type checking" which you'll need > to do anyhow. This example is no worse than an O(n) assertion. > > I can't, off the top of my head, think of a case when this occurs "more > subtly" but we will have to watch out for those. I'm with Paul on this one. Use the .pyo forms if you want the speed. During development, you'll just have to deal. In production, all the type assertions are skipped or not even compiled into the code. Runtime type assertions only occur on function entry and when the type-assert operator is used. Depending on what types of parameterization is possible and how it is implemented, there may be some asserts in there. Cheers, -g -- Greg Stein, http://www.lyra.org/ From guido@CNRI.Reston.VA.US Fri Jan 21 14:57:32 2000 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Fri, 21 Jan 2000 09:57:32 -0500 Subject: [Types-sig] namespaces In-Reply-To: Your message of "Fri, 21 Jan 2000 06:19:05 PST." References: Message-ID: <200001211457.JAA28979@eric.cnri.reston.va.us> [Greg Stein] > '!' can *definitely* invoke a compile-time check: > > x = 5 > y = x!string # Boom! > > The operator also provides information to the type-checker. Greg, you've used this phrase before, and I wonder if we're on the same sheet here. Yes, of course it provides information to the type checker -- just as any other operator does. If the type checker sees 1+1 (or i+j with i and j declared as ints), it will know that the result is an int. Of course the expression (x ! T) has type T to the type checker. This seems so obvious that I'm wondering if there's another sense in which you mean this? (Especially since at some point you thought it would be a good idea if x ! int gave a warning if the typechecker saw that x could be some other type than x -- which for me is the very reason of its existence as a dynamic cast operator.) > I agree. I posed to Guido a while back that I do not believe we can use > isinstance() to propertly typecheck the following code: > > if isinstance(x, int): > return x + 5 > if isinstance(x, str): > return x + 'hi' > > Specifically, because the two parameters are arbitrary expressions, it > will be reasonably complicated to understand the type specified by the > second argument, and to determine that the first argument is an lvalue > (meaning we can record the type on the lvalue). Well, we won't have to guarantee that we will figure it out for arbitrary expressions. We can make isinstance an operator (reserved word) and require that the left argument is a variable (maybe a dotted name) and the right argument is a type expression evaluatable at compile time. Or we can say that the type checker only uses the information conveyed by isinstance if these conditions are met. > Moreover, it is difficult to "temporarily" record "int" for "x" > within the if: block (up to that point, we think x is int|str). This seems to be an argument based on your current idea of how a typechecker should be implemented. I don't think it's difficult at all (and besides, you said difficult, not impossible). > Strictly: yes, we could typecheck it (IFF the user followed predefined > patterns and restrictions). Would I want to write the code? Gack. No :-) > > I'd rather avoid those restrictions and rely on a clean mechanism. > (described below) > > > > Are there any other ideas or > > workarounds with these ideas that address these problems? > > I am advocating against a new typecase statement. > > Specifically, Python does not have a switch statement right now for > reasons "X" and "Y" (exercise for the reader). I think those same reasons > could be applied to a typecase statement. Here is Guido's example, and my > interpretation: > > decl x: int | string | None | [any] | ... > > typecase x: > int, i: print "an int: %d" % i > string, s: print "a string: %s" % `s` > None: print "nothing" > else: print "don't know" > > if isinstance(x, int): > print "an int: %d" % (x!int) > elif isinstance(x, string): > print "a string: %s" % `x!string` > elif isinstance(x, None): > print "nothing" > else: > print "don't know" My problem with this is that in this version each type check is executed twice: first the isinstance() call, then the ! operator. Because (in your world :-) the code generator isn't smart enough to understand the meaning of the isinstance() call, it won't know that in x!int, x is always an int, so it will insert another runtime type check. > Of course, that is a *very* literal translation. We only need the (x!int) > form if the type-checker compares formats against input types. The > x!string is unnecessary since `` works on any input type. Using isinstance > to check for None is too heavyweight. > > But, let's use a "better" example: > > def munge(x: int|str)->int|str: > > # the typecase form: > > typecase x: > int, i: > return i + 5 > # can't use else: here, because we need the "casting" performed by > # the typecase statement to assign a value to "s" > str, s: > return s + 'hi' > > > # the Greg form: > > if isinstance(x, int): > return x!int + 5 > # this could be an else: or omitted (because of the preceding return) > if isinstance(x, str): > return x!str + 'hi' Same counter-argument: each type check will be coded twice in the bytecode. (Small nit: you alternatingly demonstrate ! as having a very low priority and having a very high priority. Please make up your mind.) > Specifically, I think we should use if/elif/else statements rather than a > typecase. In combination with the '!' operator, we successfully pass the > type-check step without errors. > > It is true that isinstance() is a bit more wordy, but I doubt the above > code patterns will be all that common. (can somebody supply some > examples?) Grep the standard library for 'type('; I found about 150 occurrences. Most of these are in tests that should be translated to using isinstance(). Quite a few are for functions or constructors taking either a file or a filename, e.g.: class SomeSpecialFile: def __init__(self, fn): if isinstance(fn, str): f = open(fn) else: f = fn ...use f... Adding types, this becomes in your idiom: def __init__(self, fn: str|file): decl f: file if isinstance(fn, str): f = open(fn!str) else: f = fn!file ...use f... With typecase: def __init__(self, fn: str|file): decl f: file typecase fn: str, fns: f = open(fns) file, fnf: f = fnf ...use f... Another common idiom is a function argument of type T | [T], for some T. > Here is what I find will be *very* common: > > decl checked_module > > import third_party_unchecked_module as tpum > > x = tpum.func() # by definition, return type is "any" > func_taking_str(x!str) # we know it is a string and need to say so > > In other words, I think the '!' will be common when dealing with unchecked > modules. I don't think it will be used all that often to narrow types down > from a set of alternatives (which is the purpose of a typecase statement). > Therefore, I'd rather avoid introducing the typecase syntax -- not enough > benefit for the cost of the additional syntax. Well, new syntax is new syntax. To me, the cost of adding ! seems just as high as the cost of adding typecase. But the real difference of opinion is our judgement about which will be more common. I expect that checked modules using unchecked modules is rare, and disambiguating union types is common; you expect the reverse. Only time will tell... --Guido van Rossum (home page: http://www.python.org/~guido/) From scott@chronis.pobox.com Fri Jan 21 14:58:24 2000 From: scott@chronis.pobox.com (scott) Date: Fri, 21 Jan 2000 09:58:24 -0500 Subject: [Types-sig] namespaces In-Reply-To: <200001211343.IAA28893@eric.cnri.reston.va.us> References: <20000121083648.A78943@chronis.pobox.com> <200001211343.IAA28893@eric.cnri.reston.va.us> Message-ID: <20000121095824.A79118@chronis.pobox.com> On Fri, Jan 21, 2000 at 08:43:55AM -0500, Guido van Rossum wrote: > [scott] > > Stack evaluation looks good, that clears up a lot for me re: runtime > > behavior. but there are problems with both ! (pushes type checking > > errors from compile time to runtime) and isinstance: it's > > grammatically ambiguous as a designator of sections of code blocks > > which may assume disambiguated alternate types. This becomes quite > > evident if you try to write a checker with a function call that is the > > runtime disambiguating agent. > > Scott, I don't understand the last two sentences. Can you give an > example? Just working with the existing grammar for python, specifically, it's definition of 'test': in the typical case, you want to disambiguate for the purpose of checking a suite: if x is None: suite which is gramatically if test: suite we know that test can contain any number of elements: and'd or'd, not'd, compared, unary op'd, atoms with trailers, etc. if a function call is the disambiguating agent (like isinstance), then it is grammatically subject to being part of a test comprised of more pieces than just the function call: if f() and foo if f() or foo if f() != foo if foo and f() etc. So using a function call as the disambiguator allows us to write things like if isinstance(x, T) or foo: suite Is suite disambiguated? at compile time we don't know. if isinstance(x, T) > 0: suite again at compile time, we don't know if suite is disambiguated. maybe you can just assume that any comparisons, ors, ands, etc invalidate the disambiguating quality of 'test' in if test: suite, but then your compile time checker doesn't act just the same as the runtime: if isinstance(x, T) or isinstance(y, T') and isinstance(z, T''): suite what exactly does the compile time checker do with suite? OK, so maybe we can work with a disambiguating agent that produces more complex situations than 'typecase' and family, but it sure may become complex. Now, how about: if isinstance(x, GrabMeAType()): suite GrabMeAType() returns a value of TypeType type or somesuch, but the user may know that it returns something more specific. You can't do much with that at compile time. Also, we can disambiguate like this isinstance(x, T) and [do stuff with x assuming it's of type T] Also, what if someone does this: def isinstance(x, y): return 0 or this: foo = isinstance so the compile time checker, in so far as it follows a function call as a disambiguating agent, has potentially magnitudes of more work to do than a compile time checker that uses a grammatical construct as a disambiguating agent. I tried very hard to make the idea of using a builtin work as it is by far the most aesthetically pleasing way of approaching this part of type checking to me. Addressing these issues in a way that is _consistent_ with runtime behavior at compile time is in my estimation incredibly difficult, much more so than having a grammatical construct be the disambiguating agent (which itself is hard enough to warrant at least more than all the work we've all put into it). Addressing these issues in a way that is _inconsistent_ with runtime behavior at compile time seems less than satisfactory to me. >(Have you written a compiler before? It seems quite > possible to me to define either one unambiguously.) Grammatically they can of course be defined unambiguously, but the implied semantics of a builtin function call being the disambiguator are hard enough that I'd just dub them ambiguous, and I *really* don't like the fact that '!' pushes compile time errors to run time. I'm also unsure of the implications of a disambiguator that doesn't mark the end of disambiguated code. I have written various parsers on a scale smaller than python and have written an LALR parser generator. I'm not unfamiliar with computer language parsers and context free grammars, but then again, I can't exactly say I wrote the python compiler ;) scott > > > Are there any other ideas or workarounds with these ideas that > > address these problems? > > --Guido van Rossum (home page: http://www.python.org/~guido/) > > _______________________________________________ > Types-SIG mailing list > Types-SIG@python.org > http://www.python.org/mailman/listinfo/types-sig From guido@CNRI.Reston.VA.US Fri Jan 21 15:09:06 2000 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Fri, 21 Jan 2000 10:09:06 -0500 Subject: [Types-sig] namespaces In-Reply-To: Your message of "Fri, 21 Jan 2000 09:58:24 EST." <20000121095824.A79118@chronis.pobox.com> References: <20000121083648.A78943@chronis.pobox.com> <200001211343.IAA28893@eric.cnri.reston.va.us> <20000121095824.A79118@chronis.pobox.com> Message-ID: <200001211509.KAA29014@eric.cnri.reston.va.us> [scott] > Just working with the existing grammar for python, specifically, it's > definition of 'test': > > in the typical case, you want to disambiguate for the purpose of > checking a suite: > > if x is None: suite > which is gramatically > if test: suite > > we know that test can contain any number of elements: and'd or'd, not'd, > compared, unary op'd, atoms with trailers, etc. > > if a function call is the disambiguating agent (like isinstance), then > it is grammatically subject to being part of a test comprised of more > pieces than just the function call: > > if f() and foo > if f() or foo > if f() != foo > if foo and f() > > etc. > > So using a function call as the disambiguator allows us to write > things like > > if isinstance(x, T) or foo: suite > > Is suite disambiguated? at compile time we don't know. > > if isinstance(x, T) > 0: suite > > again at compile time, we don't know if suite is disambiguated. > > maybe you can just assume that any comparisons, ors, ands, etc > invalidate the disambiguating quality of 'test' in if test: suite, but > then your compile time checker doesn't act just the same as the > runtime: > > if isinstance(x, T) or isinstance(y, T') and isinstance(z, T''): suite > > what exactly does the compile time checker do with suite? OK, so > maybe we can work with a disambiguating agent that produces more > complex situations than 'typecase' and family, but it sure may become > complex. Nothing to worry about. Language designers routinely specify exactly which kinds of expressions can be evaluated at compile time and which ones can't. (E.g. C allows expressions as array bounds but requires that they can be evaluated at compile time.) > Now, how about: > > if isinstance(x, GrabMeAType()): suite > > GrabMeAType() returns a value of TypeType type or somesuch, but the > user may know that it returns something more specific. You can't do > much with that at compile time. Is it really that hard to explain to users that using dynamic information disables the runtime typecheck? > Also, we can disambiguate like this > isinstance(x, T) and [do stuff with x assuming it's of type T] > > Also, what if someone does this: > def isinstance(x, y): return 0 > > or this: > foo = isinstance > > so the compile time checker, in so far as it follows a function call > as a disambiguating agent, has potentially magnitudes of more work to > do than a compile time checker that uses a grammatical construct as > a disambiguating agent. Still no biggie in my book -- if this is the only argument against isinstance(), and otherwise it was perfect, I am more than willing to do the work. > I tried very hard to make the idea of using a builtin work as it is by > far the most aesthetically pleasing way of approaching this part of > type checking to me. Addressing these issues in a way that is > _consistent_ with runtime behavior at compile time is in my estimation > incredibly difficult, much more so than having a grammatical construct > be the disambiguating agent (which itself is hard enough to warrant > at least more than all the work we've all put into it). Addressing > these issues in a way that is _inconsistent_ with runtime behavior at > compile time seems less than satisfactory to me. Can you explain what you mean by consistency with runtime behavior? The code generator and the type checker have to conspire to generate good code, so the code generator will by definition (have to) generate code that matches the compile time interpretation. > >(Have you written a compiler before? It seems quite > > possible to me to define either one unambiguously.) > > Grammatically they can of course be defined unambiguously, but the > implied semantics of a builtin function call being the disambiguator > are hard enough that I'd just dub them ambiguous, and I *really* don't > like the fact that '!' pushes compile time errors to run time. I'm > also unsure of the implications of a disambiguator that doesn't mark > the end of disambiguated code. From "hard" to "ambiguous" is a non-sequitur to me. You haven't pointed out any ambiguities in the semantics, only things you're not sure you know how to implement efficiently. > I have written various parsers on a scale smaller than python and have > written an LALR parser generator. I'm not unfamiliar with computer > language parsers and context free grammars, but then again, I can't > exactly say I wrote the python compiler ;) But-I-can-ly y'rs, --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@CNRI.Reston.VA.US Fri Jan 21 15:27:56 2000 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Fri, 21 Jan 2000 10:27:56 -0500 Subject: [Types-sig] Static typing: Towards closure? In-Reply-To: Your message of "Fri, 21 Jan 2000 06:35:19 PST." References: Message-ID: <200001211527.KAA29048@eric.cnri.reston.va.us> I finally understand what Greg means when he says "the ! operator gives infornation to the typechecker". In this example: > decl a: int|str > decl b: int > a = 1 > foo() > # what is the type of a? > > b = a # should fail > b = a!int # no problem > > # a is an int again > b = a the final comment gives us a clue. Greg wants us to believe that the following code should be accepted by the type checker: decl a: int|str decl b: int a!int # "tell the checker a is really an int; exception if we lied" b = a # not an error according to Greg I personally think this is a slippery path that we should not enter at all. What if another thread modifies a before we get to grab it? What if we call some function that changes it? Other questions: what if we have an expression designating an item of a dictionary, how would we propagate that information? I think that *if* we decide to have x!T in our language, the proper definiton for it should be that it raises an exception if the x doesn't have type T, and otherwise it yields an expression with type T and value x; it should have no other effects on namespaces or knowledge of the typechecker. (The optimizer is a different point, but as far as the definition of the language semantics is concerned, there is no optimizer.) Thus, to make that code pass the type checker, we would have to write b = a!int # a's type is still int|str > "projecting" a type forward from an isinstance() (as you demo above) or > within a typecase suite would have similar problems. > > typecase a: > int: > # type-checker labels "a" as an "int" > b = a # passes > foo() > # what is the type of a? > > > isinstance and '!' seem to be much more clear and robust. No, the only robust variant is the typecase syntax I proposed later: typecase a: int, ai: # ai has type int and value a b = ai # passes # b = a # would be an error foo() # ai still has type int > And a reminder for ourselves :-) ... we should be careful about labeling a > global (at all and/or across a func call). I guess by labelling you mean using something like g!T, right? --Guido van Rossum (home page: http://www.python.org/~guido/) From gstein@lyra.org Fri Jan 21 15:43:54 2000 From: gstein@lyra.org (Greg Stein) Date: Fri, 21 Jan 2000 07:43:54 -0800 (PST) Subject: [Types-sig] namespaces In-Reply-To: <200001211457.JAA28979@eric.cnri.reston.va.us> Message-ID: On Fri, 21 Jan 2000, Guido van Rossum wrote: > [Greg Stein] > > '!' can *definitely* invoke a compile-time check: > > > > x = 5 > > y = x!string # Boom! > > > > The operator also provides information to the type-checker. > > Greg, you've used this phrase before, and I wonder if we're on the > same sheet here. We are. I just keep hearing that "it only does runtime" or whatever, and respond to that. When I first raised the issue of the operator, nobody seemed to understand me when I described the compile-time aspects, so it was being dismissed as not-useful in a static checker. After I while, I think that I found the right way to explain its behavior, but I still have that reflex built in :-) >... > Well, we won't have to guarantee that we will figure it out for > arbitrary expressions. I do agree with this. I have to :-) because I think we should be able to do things like: MyType = some_function() def foo(x: MyType): ... > We can make isinstance an operator (reserved > word) and require that the left argument is a variable (maybe a dotted > name) and the right argument is a type expression evaluatable at > compile time. Or we can say that the type checker only uses > the information conveyed by isinstance if these conditions are met. It is this last part that has me squeamish. "If you do this and that, and also this, then you get the benefit." Yes, I understand that we can definitely use the isinstance() pattern (as an operator or remaining as a function) to correctly disambiguate. But I'm just a bit uncomfortable with the rules that would need to be followed. The "magic" that I was referring to. If you actually alter the syntax to enforce something like: isinstance_expr: 'isinstance' '(' dotted_name ',' typedecl ')' Then we would avoid "Wrong Forms" and people wouldn't have to remember particular rules (to me, it is easier to remember syntax than to remember syntax plus a set of rules for using that syntax) > > Moreover, it is difficult to "temporarily" record "int" for "x" > > within the if: block (up to that point, we think x is int|str). > > This seems to be an argument based on your current idea of how a > typechecker should be implemented. I don't think it's difficult at > all (and besides, you said difficult, not impossible). Just "difficult"... yes. I could certainly build it, and it might even be reasonably easy because I want to correctly handle: # no declarations if whatever: a = 1 else: a = '1' # merge the types, label 'a' as int|str In this sense, we are tracking separate type-sets for each suite. Labeling "x" as an "int" for a branch might simply be altering the "initial" type-set for each branch. >... > > if isinstance(x, int): > > print "an int: %d" % (x!int) > > elif isinstance(x, string): > > print "a string: %s" % `x!string` > > elif isinstance(x, None): > > print "nothing" > > else: > > print "don't know" > > My problem with this is that in this version each type check is > executed twice: first the isinstance() call, then the ! operator. Agreed. > Because (in your world :-) :-) > the code generator isn't smart enough to > understand the meaning of the isinstance() call, it won't know that in > x!int, x is always an int, so it will insert another runtime type > check. Correct. However, a smart one *would* know and could omit the runtime check. That option is not foreclosed by my proposal. If you believe that it is "not hard" to do the type handling, then I'll be happy with that... it helps to support my proposal :-) We would end up with just one check per branch; just like the typecase-statement proposal. >... > > if isinstance(x, int): > > return x!int + 5 > > # this could be an else: or omitted (because of the preceding return) > > if isinstance(x, str): > > return x!str + 'hi' > > Same counter-argument: each type check will be coded twice in the > bytecode. Right. > (Small nit: you alternatingly demonstrate ! as having a very low > priority and having a very high priority. Please make up your mind.) Just call me feeble :-) For some reason, I brain-farted and thought the '+' would terminate the typedecl and create an implied left-grouping. But no... instead we get x!(int + 5) and the '+' is invalid in a typedecl. > > Specifically, I think we should use if/elif/else statements rather than a > > typecase. In combination with the '!' operator, we successfully pass the > > type-check step without errors. > > > > It is true that isinstance() is a bit more wordy, but I doubt the above > > code patterns will be all that common. (can somebody supply some > > examples?) > > Grep the standard library for 'type('; I found about 150 occurrences. > Most of these are in tests that should be translated to using > isinstance(). Quite a few are for functions or constructors taking > either a file or a filename, e.g.: Hrm. Good point. >... > > Therefore, I'd rather avoid introducing the typecase syntax -- not enough > > benefit for the cost of the additional syntax. > > Well, new syntax is new syntax. To me, the cost of adding ! seems > just as high as the cost of adding typecase. Hunh. A single, new operator seemed simpler to me than the typecase statement, its branches, etc. > But the real difference of opinion is our judgement about which will > be more common. I expect that checked modules using unchecked modules > is rare, and disambiguating union types is common; you expect the > reverse. Only time will tell... True. The condition I had in mind is using a third-party library. If I sent you my httplib-in-development, then I probably don't have type annotations in it yet. However, you're going to drop it into your app to test. Grabbing modules from here and there, they may not be checked. But yah... you could also argue they would have checks added to them before release. But what about developers that don't care to do so? Cheers, -g -- Greg Stein, http://www.lyra.org/ From scott@chronis.pobox.com Fri Jan 21 15:56:30 2000 From: scott@chronis.pobox.com (scott) Date: Fri, 21 Jan 2000 10:56:30 -0500 Subject: [Types-sig] namespaces In-Reply-To: <200001211509.KAA29014@eric.cnri.reston.va.us> References: <20000121083648.A78943@chronis.pobox.com> <200001211343.IAA28893@eric.cnri.reston.va.us> <20000121095824.A79118@chronis.pobox.com> <200001211509.KAA29014@eric.cnri.reston.va.us> Message-ID: <20000121105630.C79118@chronis.pobox.com> On Fri, Jan 21, 2000 at 10:09:06AM -0500, Guido van Rossum wrote: [...] > Nothing to worry about. Language designers routinely specify exactly > which kinds of expressions can be evaluated at compile time and which > ones can't. (E.g. C allows expressions as array bounds but requires > that they can be evaluated at compile time.) Sounds good. Any ideas of what those specs might be for isinstance? > > > Now, how about: > > > > if isinstance(x, GrabMeAType()): suite > > > > GrabMeAType() returns a value of TypeType type or somesuch, but the > > user may know that it returns something more specific. You can't do > > much with that at compile time. > > Is it really that hard to explain to users that using dynamic > information disables the runtime typecheck? You mean compile time check right? Hopefully not. It's just an example of a problem with isinstance that doesn't exist with an explicit grammatical construct. > > > Also, we can disambiguate like this > > isinstance(x, T) and [do stuff with x assuming it's of type T] > > > > Also, what if someone does this: > > def isinstance(x, y): return 0 > > > > or this: > > foo = isinstance > > > > so the compile time checker, in so far as it follows a function call > > as a disambiguating agent, has potentially magnitudes of more work to > > do than a compile time checker that uses a grammatical construct as > > a disambiguating agent. > > Still no biggie in my book -- if this is the only argument against > isinstance(), and otherwise it was perfect, I am more than willing to > do the work. It's the argument that prevented me from going with it. Working on the assumption that the compile time semantics must be identical to the run time semantics makes it quite hard. FWIW, I'm more than willing to help do the work. > > > I tried very hard to make the idea of using a builtin work as it is by > > far the most aesthetically pleasing way of approaching this part of > > type checking to me. Addressing these issues in a way that is > > _consistent_ with runtime behavior at compile time is in my estimation > > incredibly difficult, much more so than having a grammatical construct > > be the disambiguating agent (which itself is hard enough to warrant > > at least more than all the work we've all put into it). Addressing > > these issues in a way that is _inconsistent_ with runtime behavior at > > compile time seems less than satisfactory to me. > > Can you explain what you mean by consistency with runtime behavior? Consistency in that there's no need for a spec (other than the grammar) about exactly in what contexts isinstance is evaluated at compile time. I suppose that if such a spec were made very clear and didn't have too steep a learning curve in terms of using isinstance, I may well find it satisfactory. isinstance looks a lot like existing ways to check types in current python, and that's well worth striving for. scott From gstein@lyra.org Fri Jan 21 16:07:39 2000 From: gstein@lyra.org (Greg Stein) Date: Fri, 21 Jan 2000 08:07:39 -0800 (PST) Subject: [Types-sig] how to do a typecase? (was: Static typing: Towards closure?) In-Reply-To: <200001211527.KAA29048@eric.cnri.reston.va.us> Message-ID: On Fri, 21 Jan 2000, Guido van Rossum wrote: > I finally understand what Greg means when he says "the ! operator > gives infornation to the typechecker". In this example: hehe... see? I still am not clear sometimes. >... > the final comment gives us a clue. Greg wants us to believe that the > following code should be accepted by the type checker: > > decl a: int|str > decl b: int > > a!int # "tell the checker a is really an int; exception if we lied" > b = a # not an error according to Greg > > I personally think this is a slippery path that we should not enter at > all. What if another thread modifies a before we get to grab it? > What if we call some function that changes it? Other questions: what > if we have an expression designating an item of a dictionary, how > would we propagate that information? All these are true, and are mostly related to the fact that "a" is a global in this case. If you're talking about locals, then the above code could be fine. But your point about dictionaries... example: x = y[5] ! int # what do we know now? Actually, you're just throwing an issue back at me, when I complained about: if isinstance(y[5], int): :-) In other words, 'a!int' *can* say something about a, but the type-checker will have to apply various restrictions about what it can accept for that left-hand operand (and still be able to extract meaning). And if those rules need to be part of the language definition, then I would retract the issue. (since I'd rather avoid the rules thing...) > I think that *if* we decide to have x!T in our language, the proper > definiton for it should be that it raises an exception if the x > doesn't have type T, and otherwise it yields an expression with type T > and value x; it should have no other effects on namespaces or > knowledge of the typechecker. (The optimizer is a different point, > but as far as the definition of the language semantics is concerned, > there is no optimizer.) > > Thus, to make that code pass the type checker, we would have to write > > b = a!int > # a's type is still int|str I would agree with these two statements. In certain cases, the code at the top of this email could be accepted, but to wire that into the definition would imply too many rules and restrictions (instead of the simple definition for '!' that you just provided). And relegating the problem to the optimizer to remove the runtime checks is perfectly fine with me. > > "projecting" a type forward from an isinstance() (as you demo above) or > > within a typecase suite would have similar problems. > > > > typecase a: > > int: > > # type-checker labels "a" as an "int" > > b = a # passes > > foo() > > # what is the type of a? Note: this was just a demo, regarding the potential changes to "a" by the foo() function. I do agree that the "int, ai" variant is more robust against these types of changes. > > isinstance and '!' seem to be much more clear and robust. > > No, the only robust variant is the typecase syntax I proposed later: > > typecase a: > int, ai: > # ai has type int and value a > b = ai # passes > # b = a # would be an error > foo() > # ai still has type int Hrm. True. Even with threading present, the object referenced by "a" can be pushed on the stack (thus: safe from changes to "a"). The typecase is certainly looking more reasonable to me :-). Especially based on your quick review of the standard library. > > And a reminder for ourselves :-) ... we should be careful about labeling a > > global (at all and/or across a func call). > > I guess by labelling you mean using something like g!T, right? Sorry... bad wording. I meant we should be careful how the type-checker records type information for a name. In my prototype checker, I have been setting it up to automatically create a union of the types seen for a variable. For example: # no decl a = 1 foo() a = 'a' bar() For checking purposes, "a" would have int|str. It is somewhat questionable what the type should be as the global code is being checked; certainly it would be int|str for the checks of the function bodies. Cheers, -g -- Greg Stein, http://www.lyra.org/ From scott@chronis.pobox.com Fri Jan 21 16:05:12 2000 From: scott@chronis.pobox.com (scott) Date: Fri, 21 Jan 2000 11:05:12 -0500 Subject: [scott@chronis.pobox.com: Re: [Types-sig] namespaces] Message-ID: <20000121110511.D79118@chronis.pobox.com> Forgot to send this on to the list again :) ----- Forwarded message from scott ----- Date: Fri, 21 Jan 2000 10:07:34 -0500 From: scott To: Greg Stein Subject: Re: [Types-sig] namespaces X-Mailer: Mutt 0.95.7i In-Reply-To: Forwarded to list [edited slightly] On Fri, Jan 21, 2000 at 06:19:05AM -0800, Greg Stein wrote: > On Fri, 21 Jan 2000, scott wrote: > > Stack evaluation looks good, that clears up a lot for me re: runtime > > behavior. but there are problems with both ! (pushes type checking > > errors from compile time to runtime) > > '!' can *definitely* invoke a compile-time check: > > x = 5 > y = x!string # Boom! > can or always will? can is easy and doesn't translate to anything beyond what could be done with a type checker that disallows alternate types. You're definition of ! states that it can raise a runtime error. In the cases where that can happen, it doesn't serve the purpose of static type checking. The extent to which it does raise runtime errors is exactly what is incomplete and incompletable about a type checking system that uses it to disambiguate alternates. scott ----- End forwarded message ----- From guido@CNRI.Reston.VA.US Fri Jan 21 16:33:45 2000 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Fri, 21 Jan 2000 11:33:45 -0500 Subject: [Types-sig] how to do a typecase? (was: Static typing: Towards closure?) In-Reply-To: Your message of "Fri, 21 Jan 2000 08:07:39 PST." References: Message-ID: <200001211633.LAA29205@eric.cnri.reston.va.us> > From: Greg Stein > In certain cases, the code at the top of this email could be accepted, but > to wire that into the definition would imply too many rules and > restrictions (instead of the simple definition for '!' that you just > provided). It is *not* acceptable that whether a program is considered typesafe depends on particulars of the typechecker implementation (except where limits on program size etc. are concerned). Saying that a particular construct is defined as not typesafe, but that a smarter typechecker might allow it anyway if it sees that in this case there are no problems, is asking for trouble: people using the smarter typechecker will innocently publish code that they think is typesafe while actually it isn't. True, this is always a risk when you have multiple implementations of the same set of complex rules, but we shouldn't build this attitude into our definition of the typechecker. > And relegating the problem to the optimizer to remove the runtime checks > is perfectly fine with me. That's of course the solution. --Guido van Rossum (home page: http://www.python.org/~guido/) From jeremy@cnri.reston.va.us Fri Jan 21 17:19:29 2000 From: jeremy@cnri.reston.va.us (Jeremy Hylton) Date: Fri, 21 Jan 2000 12:19:29 -0500 (EST) Subject: [Types-sig] how to do a typecase? (was: Static typing: Towards closure?) In-Reply-To: <200001211633.LAA29205@eric.cnri.reston.va.us> References: <200001211633.LAA29205@eric.cnri.reston.va.us> Message-ID: <14472.38177.65905.81604@goon.cnri.reston.va.us> >>>>> "GvR" == Guido van Rossum writes: GvR> True, this is always a risk when you have multiple GvR> implementations of the same set of complex rules, but we GvR> shouldn't build this attitude into our definition of the GvR> typechecker. Absolutely! The rules may be complex but they should be stated precisely and unambiguously. The implementation of the type checker will be complex, so we need these rules to verify the correctness of the type checker. If a program checks out with one implementation and not with another, then one of the type checkers has a bug! Jeremy From paul@prescod.net Fri Jan 21 16:51:24 2000 From: paul@prescod.net (Paul Prescod) Date: Fri, 21 Jan 2000 08:51:24 -0800 Subject: [Types-sig] namespaces (was: Changing existing class instances) References: Message-ID: <38888E8C.FE369A86@prescod.net> Greg Stein wrote: > Tim Peters: > > Trying to overload the current namespace set makes it so > > much harder to see that these are compile-time gimmicks, and users need to > > be acutely aware of that if they're to use it effectively. Note that I > > understand (& wholly agree with) the need for runtime introspection. > > And that is the crux of the issue: I think the names that are assigned to > these classes, interfaces, typedefs, or whatever, can follow the standard > Python semantics and be plopped into the appropriate namespace. There is > no overloading. This is indeed the crux of the issue. For those that missed it last time, it became very clear to me that we are working with radically different design aesthetics when we discussed the idea of having an optional keyword that said: "this thing is usually handled at compile time but I want to handle it at runtime. I know what I am doing." Greg complained that that would require the programmer to understand too much what was being done at compile time and what at runtime and what. >From my point of view this is *exactly* what a programmer *needs* to know and if we make it too hard for them to know it then we have failed. > There is no "overloading" of namespaces. We are using Python namespaces > just like they should be, and the type-checker doesn't even have to be all > the smart to track this stuff. There is an overloading of namespaces because we will separately specify the *compile time semantics* of these names. We need to separately specify these semantics because we need all compile time type checkers to behave identically. Yes, it seems elegant to make type objects seem as if they are "just like" Python objects. Unfortunately they aren't. Type objects are evaluated -- and accepted or rejected -- at compile time. Every programmer needs to understand that and it should be blatantly obvious in the syntax, just as everything else in Python syntax is blatantly obvious. -- Paul Prescod - ISOGEN Consulting Engineer speaking for himself Earth will soon support only survivor species -- dandelions, roaches, lizards, thistles, crows, rats. Not to mention 10 billion humans. - Planet of the Weeds, Harper's Magazine, October 1998 From skip@mojam.com (Skip Montanaro) Fri Jan 21 17:58:49 2000 From: skip@mojam.com (Skip Montanaro) (Skip Montanaro) Date: Fri, 21 Jan 2000 11:58:49 -0600 (CST) Subject: [Types-sig] Static typing: Towards closure? In-Reply-To: <14471.41161.944820.970237@bitdiddle.cnri.reston.va.us> References: <200001191722.MAA19286@eric.cnri.reston.va.us> <20000120231625.A899@vet.uu.nl> <200001202231.RAA27334@eric.cnri.reston.va.us> <20000121004112.B987@vet.uu.nl> <14471.41161.944820.970237@bitdiddle.cnri.reston.va.us> Message-ID: <14472.40537.978169.76307@beluga.mojam.com> Jeremy> Guido showed the potentially painful example of passing a list Jeremy> containing thousands of elements from unchecked code to an Jeremy> object declared as "def (int list) -> int"; this seems to imply Jeremy> a runtime check to verify that all of the elements of the list Jeremy> are ints. I worry that there would be subtler cases with Jeremy> similar performance hits. I think this is an excellent example of where type propagation might rear its ugly head. How do you get rid of the performance hit? If you don't want to (or can't) rewrite your code, you'll add type declarations to it. If a call from an unchecked module into a checked module occurs and causes a protective performance hit as you suggest, my first inclination would be to add types to the unchecked module, especially if the checked module was out of my control (installed with type checking by the sys admin on a shared server, for example). A quick example, presuming the file object (in whatever guise - will C modules be expected to be type-safe against calls from unchecked Python modules?) acquires type checking and I make the dumb mistake of executing f = open("/tmp/foo", "w") f.writelines(["x\n"] * 100000) then the file object's writelines method will protect itself by verifying that each list element is a string. I suspect my only recourses here would be to rewrite the calling code (maybe not as easy as in this contrived example) or add type checking to the previously unchecked module. That would then implicitly define a new set of untyped/type interfaces that may suffer from similar performance problems and have to be addressed somehow. Skip From skip@mojam.com (Skip Montanaro) Fri Jan 21 18:05:10 2000 From: skip@mojam.com (Skip Montanaro) (Skip Montanaro) Date: Fri, 21 Jan 2000 12:05:10 -0600 (CST) Subject: [Types-sig] Static typing: Towards closure? In-Reply-To: <20000121013812.A1120@vet.uu.nl> References: <200001191722.MAA19286@eric.cnri.reston.va.us> <20000120231625.A899@vet.uu.nl> <200001202231.RAA27334@eric.cnri.reston.va.us> <20000121004112.B987@vet.uu.nl> <200001210001.TAA28120@eric.cnri.reston.va.us> <20000121013812.A1120@vet.uu.nl> Message-ID: <14472.40918.847126.312724@beluga.mojam.com> Martijn> Note that runtime checking might help avoid the list of 1000 Martijn> ints problem in some cases; this list can be declared to be of Martijn> ints and this can be checked at runtime, so that passing it Martijn> into checked function can proceed without overhead. But, if Martijn> this kind of thing is allowed, you could very well have a Martijn> type-infection problem; in order to avoid the overhead people Martijn> are pushed to add checks everywhere. On which side of the function call is checking going to be performed? If it's in the called (and presumably typed) function, the person compiling the calling code can't do anything to disable it short of recompiling the called module with type checking disabled (which may not be an option). And what about extension modules? Will -O execution disable them? Skip Montanaro | http://www.mojam.com/ skip@mojam.com | http://www.musi-cal.com/ 847-971-7098 From jeremy@cnri.reston.va.us Fri Jan 21 18:03:10 2000 From: jeremy@cnri.reston.va.us (Jeremy Hylton) Date: Fri, 21 Jan 2000 13:03:10 -0500 (EST) Subject: [Types-sig] lambdas, classes, tuples, default arguments, map In-Reply-To: References: Message-ID: <14472.40798.243306.210590@goon.cnri.reston.va.us> >>>>> "TL" == Tony Lownds writes: TL> 5. map has a function signature that is pretty hard to declare, TL> even with parameterization of functions. I hope it will be TL> special-cased rather than under-specified. We can write functions like map in dynamically typed Python, but I don't have much hope that we'll be able to write statically checkable types for these functions. (Happy to be proved wrong.) The hard problem is the combination of *args and polymorphic types. The type of one argument implies some constraint on the *number* or types of the other arguments. (I think Tim posted an extended example of the problem for map, but I can't find it at the moment.) If we can't write statically check these types, then it appears that any module that includes them can never be a statically checked module. (Assuming for the moment Guido's definition: "The Python implementation should guarantee that a checked module cannot raise TypeError and similar exceptions...") This leads to my worry that performance could be worse with partial static typing that it is with all runtime typechecks. Jeremy From Vladimir.Marangozov@inrialpes.fr Fri Jan 21 19:14:40 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Fri, 21 Jan 2000 20:14:40 +0100 (CET) Subject: [Types-sig] Static typing: Towards closure? (fwd) Message-ID: <200001211914.UAA03448@python.inrialpes.fr> rwarded message: From Vladimir.Marangozov@inrialpes.fr Fri Jan 21 19:12:05 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir.Marangozov@inrialpes.fr) Date: Fri, 21 Jan 2000 20:12:05 +0100 (CET) Subject: [Types-sig] Static typing: Towards closure? In-Reply-To: <200001191722.MAA19286@eric.cnri.reston.va.us> from "Guido van Rossum" at Jan 19, 2000 12:22:12 PM Bah. Forgot to cc the list, sorry. ---------------------------------------------------------------------------- Subject: Re: [Types-sig] Static typing: Towards closure? To: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Fri, 21 Jan 2000 20:12:05 +0100 (CET) Reply-To: Vladimir.Marangozov@inrialpes.fr Guido van Rossum wrote: > > I've had an off-line discussion with Greg Stein (mostly) and Paul > Prescod about trying to converge on a single proposal. Thanks for the proposal. It contains a lot of worthy points that I need to digest. BTW, I found the slides quite helpful in building an overall picture of the proposal. Without going into the details, here's my overall impression: 1. If static type checking pretends to be optional, you'd really, really, really want to forget the inline form and remain with the explicit one. (at least as a start). I understand that type declarations tend to "penetrate" the code in order to allow more checks, but if we allow this, I predict that the end of the story will be that we'll silently cross the point of no return where Python code will look like C++. And I'll regret this. At some point in the future, you'll regret it too. Retaining only the explicit form of type declarations will keeps us on the safe side, preventing us to "pollute" the main code with "optional" constructs. This will also defer the whole discussion on 'decl: def' shortcuts and the like. Let's keep the original code (the dynamic/unchecked one) clean! I definitely perceive the inline form as Python code pollution and believe that it is potentially unappropriate in a number of areas. If you still like the inline notation, please make sure that you build a tool first, which can convert automatically all inline typedecls into explicit ones. If the tool cannot handle an inline decl, you've crossed the critical point, and "optional" won't hold anymore. 2. I like the : notation. It seems concise unless you see Guido's slide No. 8 :-), where there's an example of a typed dict. It's not very clear which colon is part of the proposed notation and which one separates the text from the examples. (Guido: i'd suggest changing the color or font style for the examples in the whole presentation) 3. After reading the proposal 2 times, I still feel uncomfortable with the "decl" keyword. This term is too generic. Python is a declarative language, class and def are defacto declarations (with a default type of "any"), etc. In this context, "decl" is not very clear and the newbie following a CP4E course would have to read the docs, not the source, to understand that "decl" means a type declaration. It seemed to me that in a number of places in the proposal, "decl" is not sound at all and that it may well be replaced by "typedecl" or simply "typed" (abbrev. of "type declaration" ;-). Thus, a typed module would start with the "typed" keyword alone, which is more logical than "decl" alone. Also, typed var declarations seem to be easily readable for me: typed i : int typed m : str, n : str typed f : def(int, {str : {str : str}}) -> dict (???) typed kw_type = {str : {str : str}} typed f : def(int, kw_type) -> dict (???) is this correct This is simply a matter of clarity. I won't push too much for "typed" though, because static type checking in Python is, of course, optional :-). -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From tony@metanet.com Fri Jan 21 19:25:03 2000 From: tony@metanet.com (Tony Lownds) Date: Fri, 21 Jan 2000 11:25:03 -0800 (PST) Subject: [Types-sig] lambdas, classes, tuples, default arguments, map In-Reply-To: <14472.40798.243306.210590@goon.cnri.reston.va.us> Message-ID: On Fri, 21 Jan 2000, Jeremy Hylton wrote: > >>>>> "TL" == Tony Lownds writes: > > TL> 5. map has a function signature that is pretty hard to declare, > TL> even with parameterization of functions. I hope it will be > TL> special-cased rather than under-specified. > > We can write functions like map in dynamically typed Python, but I > don't have much hope that we'll be able to write statically checkable > types for these functions. (Happy to be proved wrong.) I dont have much hope that we will be able to write the true type of map using decl statements either. But I think the builtin map could still be type checked, as long as it is seen as a special case. I'm not too worried about type checking a user's functions that have a similarly hard signature, just map. # here is an approximation of map decl map1 = def(def(T1)->R, [T1]) -> [R] decl map2 = def(def(T1,T2)->R, [T1], [T2]) -> [R] decl map3 = def(def(T1,T2,T3)->R, [T1], [T2], [T3]) -> [R] decl map_signature = map1 & map2 & map3 decl map : map_signature -Tony From guido@CNRI.Reston.VA.US Fri Jan 21 19:34:35 2000 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Fri, 21 Jan 2000 14:34:35 -0500 Subject: [Types-sig] lambdas, classes, tuples, default arguments, map In-Reply-To: Your message of "Fri, 21 Jan 2000 11:25:03 PST." References: Message-ID: <200001211934.OAA29597@eric.cnri.reston.va.us> [Tony Lownds] > I dont have much hope that we will be able to write the true type of map > using decl statements either. But I think the builtin map could > still be type checked, as long as it is seen as a special case. I'm not > too worried about type checking a user's functions that have a similarly > hard signature, just map. Agreed. > # here is an approximation of map > decl map1 = def(def(T1)->R, [T1]) -> [R] > decl map2 = def(def(T1,T2)->R, [T1], [T2]) -> [R] > decl map3 = def(def(T1,T2,T3)->R, [T1], [T2], [T3]) -> [R] > decl map_signature = map1 & map2 & map3 > decl map : map_signature Very good -- you noticed that this requires the use of '&', not '|', to combine the constituent types! Of course. --Guido van Rossum (home page: http://www.python.org/~guido/) From paul@prescod.net Fri Jan 21 20:14:13 2000 From: paul@prescod.net (Paul Prescod) Date: Fri, 21 Jan 2000 12:14:13 -0800 Subject: [Types-sig] Static typing: Towards closure? References: <200001191722.MAA19286@eric.cnri.reston.va.us> <20000120231625.A899@vet.uu.nl> <200001202231.RAA27334@eric.cnri.reston.va.us> <20000121004112.B987@vet.uu.nl> <200001210001.TAA28120@eric.cnri.reston.va.us> <20000121013812.A1120@vet.uu.nl> <14472.40918.847126.312724@beluga.mojam.com> Message-ID: <3888BE15.CDDD4934@prescod.net> Skip Montanaro wrote: > > On which side of the function call is checking going to be performed? Is it necessarily on one side or the other? Or "between"? > If > it's in the called (and presumably typed) function, the person compiling the > calling code can't do anything to disable it short of recompiling the called > module with type checking disabled (which may not be an option). And what > about extension modules? Will -O execution disable them? Why would extension modules behave differently than they do today? -- Paul Prescod - ISOGEN Consulting Engineer speaking for himself Earth will soon support only survivor species -- dandelions, roaches, lizards, thistles, crows, rats. Not to mention 10 billion humans. - Planet of the Weeds, Harper's Magazine, October 1998 From skaller@maxtal.com.au Fri Jan 21 20:27:50 2000 From: skaller@maxtal.com.au (skaller) Date: Sat, 22 Jan 2000 07:27:50 +1100 Subject: [Types-sig] how to do a typecase? (was: Static typing: Towards closure?) References: <200001211633.LAA29205@eric.cnri.reston.va.us> <14472.38177.65905.81604@goon.cnri.reston.va.us> Message-ID: <3888C146.FCABD752@maxtal.com.au> Jeremy Hylton wrote: > > >>>>> "GvR" == Guido van Rossum writes: > > GvR> True, this is always a risk when you have multiple > GvR> implementations of the same set of complex rules, but we > GvR> shouldn't build this attitude into our definition of the > GvR> typechecker. > > Absolutely! The rules may be complex but they should be stated > precisely and unambiguously. The implementation of the type checker > will be complex, so we need these rules to verify the correctness of > the type checker. If a program checks out with one implementation and > not with another, then one of the type checkers has a bug! No. Goedel's law says that any type language sufficiently powerful to handle general recursion, which is required for general programming, cannot be type checked by any fixed algorithm. def f(): return f for example. -- John (Max) Skaller, mailto:skaller@maxtal.com.au 10/1 Toxteth Rd Glebe NSW 2037 Australia voice: 61-2-9660-0850 homepage: http://www.maxtal.com.au/~skaller download: ftp://ftp.cs.usyd.edu/au/jskaller From skaller@maxtal.com.au Fri Jan 21 20:58:33 2000 From: skaller@maxtal.com.au (skaller) Date: Sat, 22 Jan 2000 07:58:33 +1100 Subject: [Types-sig] namespaces References: <20000121083648.A78943@chronis.pobox.com> <200001211343.IAA28893@eric.cnri.reston.va.us> <20000121095824.A79118@chronis.pobox.com> <200001211509.KAA29014@eric.cnri.reston.va.us> <20000121105630.C79118@chronis.pobox.com> Message-ID: <3888C879.5A1CF3FB@maxtal.com.au> > On Fri, Jan 21, 2000 at 10:09:06AM -0500, Guido van Rossum wrote: > [...] > > Nothing to worry about. Language designers routinely specify exactly > > which kinds of expressions can be evaluated at compile time and which > > ones can't. (E.g. C allows expressions as array bounds but requires > > that they can be evaluated at compile time.) FYI: C9X allows runtime array bounds for automatic variables. -- John (Max) Skaller, mailto:skaller@maxtal.com.au 10/1 Toxteth Rd Glebe NSW 2037 Australia voice: 61-2-9660-0850 homepage: http://www.maxtal.com.au/~skaller download: ftp://ftp.cs.usyd.edu/au/jskaller From skaller@maxtal.com.au Fri Jan 21 21:01:03 2000 From: skaller@maxtal.com.au (skaller) Date: Sat, 22 Jan 2000 08:01:03 +1100 Subject: [Types-sig] namespaces (was: Changing existing class instances) References: <38888E8C.FE369A86@prescod.net> Message-ID: <3888C90F.BECA3FF6@maxtal.com.au> Paul Prescod wrote: > Yes, it seems elegant to make type objects seem as if they are "just > like" Python objects. Unfortunately they aren't. Yes they are. They even have a type, TypeType. -- John (Max) Skaller, mailto:skaller@maxtal.com.au 10/1 Toxteth Rd Glebe NSW 2037 Australia voice: 61-2-9660-0850 homepage: http://www.maxtal.com.au/~skaller download: ftp://ftp.cs.usyd.edu/au/jskaller From jeremy@cnri.reston.va.us Fri Jan 21 21:45:03 2000 From: jeremy@cnri.reston.va.us (Jeremy Hylton) Date: Fri, 21 Jan 2000 16:45:03 -0500 (EST) Subject: [Types-sig] how to do a typecase? (was: Static typing: Towards closure?) In-Reply-To: <3888C146.FCABD752@maxtal.com.au> References: <200001211633.LAA29205@eric.cnri.reston.va.us> <14472.38177.65905.81604@goon.cnri.reston.va.us> <3888C146.FCABD752@maxtal.com.au> Message-ID: <14472.54111.416650.255321@goon.cnri.reston.va.us> >>>>> "JS" == skaller writes: >>>>> "GvR" == Guido van Rossum writes: GvR> True, this is always a risk when you have multiple GvR> implementations of the same set of complex rules, but we GvR> shouldn't build this attitude into our definition of the GvR> typechecker. [Jeremy Hylton wrote:] >> Absolutely! The rules may be complex but they should be stated >> precisely and unambiguously. The implementation of the type >> checker will be complex, so we need these rules to verify the >> correctness of the type checker. If a program checks out with >> one implementation and not with another, then one of the type >> checkers has a bug! JS> No. Goedel's law says that any type language sufficiently JS> powerful to handle general recursion, which is required for JS> general programming, cannot be type checked by any fixed JS> algorithm. JS> def f(): return f JS> for example. Whoa! I don't understand how your response relates to my post. My argument restated briefly is that we should clearly specify the type system for Python, so that we can verify that typecheckers faithfully implement the type rules. Jeremy From skip@mojam.com (Skip Montanaro) Fri Jan 21 22:58:52 2000 From: skip@mojam.com (Skip Montanaro) (Skip Montanaro) Date: Fri, 21 Jan 2000 16:58:52 -0600 (CST) Subject: [Types-sig] Static typing: Towards closure? In-Reply-To: <3888BE15.CDDD4934@prescod.net> References: <200001191722.MAA19286@eric.cnri.reston.va.us> <20000120231625.A899@vet.uu.nl> <200001202231.RAA27334@eric.cnri.reston.va.us> <20000121004112.B987@vet.uu.nl> <200001210001.TAA28120@eric.cnri.reston.va.us> <20000121013812.A1120@vet.uu.nl> <14472.40918.847126.312724@beluga.mojam.com> <3888BE15.CDDD4934@prescod.net> Message-ID: <14472.58540.949119.873565@beluga.mojam.com> >> And what about extension modules? Will -O execution disable them? Paul> Why would extension modules behave differently than they do today? Consider the file writelines example again. I am not familiar with the actual code, but it's perfectly reasonable to assume that today it just starts munching its argument list, writing out lines, and will only raise an exception if/when it encounters a non-string element. The way I've been interpreting the discussion so far, checked modules would perform all their type checking *as a defense against unchecked modules* before performing any useful work, so it seems reasonable to assume that extension modules would be modified in the same way, so they would look like for each_of_my_arguments: check_its_validity now_do_what_you_pay_me_for instead of just now_do_what_you_pay_me_for which would raise errors after having generated partial output. I suppose the distinction is subtle, but it appears those are the semantics people are throwing around about checked Python modules. Should the semantics for extension modules be the same (to the extent possible)? Skip Montanaro | http://www.mojam.com/ skip@mojam.com | http://www.musi-cal.com/ 847-971-7098 From gstein@lyra.org Sat Jan 22 12:25:50 2000 From: gstein@lyra.org (Greg Stein) Date: Sat, 22 Jan 2000 04:25:50 -0800 (PST) Subject: [Types-sig] updated proposal Message-ID: I've updated my proposal some more: http://www.lyra.org/greg/python/typesys/type-proposal.html Some specific changes: - shifted all "interface" stuff to a separate section (for consideration in a future iteration of the type system) - refined the typedecl syntax - changed 'or' to '|' - added '&' - removed the ambiguity between tuple declarators and grouping (the discriminatin is now performed post-parse, just like Python's tuple-constructor/grouping syntax) - fixed ambiguity problem of: [NAME ':'] typedecl - add issue about "this proposal does not have a notion of 'const'" - other nits and updates I still have a lot of updating to do, but I need to go to sleep. I'm leaving town (for a couple days) tomorrow. While I'm gone, I'm going to bring my laptop and work on my prototype type-checker. Look for an update next Monday! You can view detailed changes to my proposal at: http://www.lyra.org/cgi-bin/viewcvs.cgi/gjspy/typesys/type-proposal.html Cheers, -g -- Greg Stein, http://www.lyra.org/ From skaller@maxtal.com.au Sat Jan 22 14:28:27 2000 From: skaller@maxtal.com.au (skaller) Date: Sun, 23 Jan 2000 01:28:27 +1100 Subject: [Types-sig] how to do a typecase? (was: Static typing: Towards closure?) References: <200001211633.LAA29205@eric.cnri.reston.va.us> <14472.38177.65905.81604@goon.cnri.reston.va.us> <3888C146.FCABD752@maxtal.com.au> <14472.54111.416650.255321@goon.cnri.reston.va.us> Message-ID: <3889BE8B.12A0618F@maxtal.com.au> Jeremy Hylton wrote: > >> Absolutely! The rules may be complex but they should be stated > >> precisely and unambiguously. The implementation of the type > >> checker will be complex, so we need these rules to verify the > >> correctness of the type checker. If a program checks out with > >> one implementation and not with another, then one of the type > >> checkers has a bug! > > JS> No. Goedel's law says that any type language sufficiently > JS> powerful to handle general recursion, which is required for > JS> general programming, cannot be type checked by any fixed > JS> algorithm. > > JS> def f(): return f > > JS> for example. > > Whoa! I don't understand how your response relates to my post. > > My argument restated briefly is that we should clearly specify the > type system for Python, so that we can verify that typecheckers > faithfully implement the type rules. And my response is: what you are calling for is not mathematically possible. You are aware of the halting problem [a restatement of Goedels theorem in a computing context]? There is an analog for type systems. The theorem says that any sufficiently rich type system cannot be checked: finding a 'complete' checker is analogous to finding an algorithm that tells if a program terminates. [Every 'checker' will fail on some cases, either explicitly, or by looping for ever.] So on the assumption that we want a rich enough type system to describe Python, it isn't reasonable to require that 'if a program checks out with one implementation and not with another, then one of the type checkers has a bug' because it is IMPOSSIBLE to construct a type checker that checks all cases. IF you want a deterministic checker, it is necessary to weaken the type system, making it less expressive (catching less errors early). One of the NICE things about python is that static type systems do not stand in the way of correct code development -- as they invariably do in every statically typed programming language in existence, on occasion. It is, in my opinion, essential to preserve this property, which is why _optional_ static typing makes sense for python. The extent to which type correctness can be determined in this scenario is the same as for any other rich static type system: no _fixed_ algorithm can do it. There is always room for improvement. I agree completely with you when you say the type system should be clearly stated. But the semantics for 'ill typed program' should be 'undefined'. If you are silly enough to specify behaviour, then you are allowing ill typed programs to have well defined meanings. (which is absurd). What this means is that you can specify a MINIMAL set of required (compile time) diagnostics, by considering what a particular algorithm can detect: a good idea is to use a 'reference implementation'. But you cannot require diagnostics in all cases, so that some algorithms will generate run time checks sometimes, when others produce compile time diagnostics, and others will optimise the code incorrectly and core dump, and others will loop forever. There is also class of programs which a translator can be required to accept, so we get the picture: 1 MUST ACCEPT (well defined) 2 MAY ACCEPT, REJECT, OR ANYTHING ELSE (undefined) 3 MUST REJECT (well defined) Ideally, classes 1 & 3 are well specified; and class (2) can be narrowed down over time with research and experience. It IS possible to eliminate (2) altogether -- by shifting the cases into (1) or (3). Consequently some correct programs will be rejected, and some incorrect ones will be accepted: the type system is then either unsound or not expressive (or both) and thus has degraded utility. -- John (Max) Skaller, mailto:skaller@maxtal.com.au 10/1 Toxteth Rd Glebe NSW 2037 Australia voice: 61-2-9660-0850 homepage: http://www.maxtal.com.au/~skaller download: ftp://ftp.cs.usyd.edu/au/jskaller From guido@CNRI.Reston.VA.US Sat Jan 22 15:41:16 2000 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Sat, 22 Jan 2000 10:41:16 -0500 Subject: [Types-sig] typecase hack Message-ID: <200001221541.KAA00535@eric.cnri.reston.va.us> Just asked me to explain the need for typecase again, and I realized that there's a parallel with except clauses. With only a little bit of imagination we could make the official way to spell a typecase as follows: try: raise x except int, xi: ...use xi: int... except str, xs: ...use xs: str... I'm not saying this is particularly elegant -- but it should work, once types and classes are more similar. --Guido van Rossum (home page: http://www.python.org/~guido/) From tim_one@email.msn.com Sat Jan 22 21:16:27 2000 From: tim_one@email.msn.com (Tim Peters) Date: Sat, 22 Jan 2000 16:16:27 -0500 Subject: [Types-sig] how to do a typecase? (was: Static typing: Towards closure?) In-Reply-To: <3889BE8B.12A0618F@maxtal.com.au> Message-ID: <000101bf651d$f21850c0$132d153f@tim> [Jeremy Hylton] > My argument restated briefly is that we should clearly specify > the type system for Python, so that we can verify that > typecheckers faithfully implement the type rules. [John Skaller] > And my response is: what you are calling for is not > mathematically possible. You are aware of the halting > problem [a restatement of Goedels theorem in a computing > context]? There is an analog for type systems. The theorem > says that any sufficiently rich type system cannot be checked: > finding a 'complete' checker is analogous to finding an > algorithm that tells if a program terminates. [Every 'checker' > will fail on some cases, either explicitly, or by looping for > ever.] Proving the *equivalence* of two distinct typecheckers is nevertheless not precluded by the (presumed(*)) unsolvability of the typechecking problem. A clean analogy is to proof procedures for first-order logic, where several distinct (sound and complete, but not effective -- it's unsolvable in general) procedures are known to be equivalent. As a ridiculous example, take a typechecker (or theorem prover) written in C and add the line "(void)42;" at the start of main(). Different program, trivially equivalent. Proving equivalence may be harder in other cases . No matter how hard the problem is, there's nothing to prevent Python from specifying *a* typechecking algorithm, then insisting that all Python typecheckers be equivalent to it. That would satisfy Jeremy's key (& reasonable!) requirement: If a program checks out with one implementation and not with another, then one of the type checkers has a bug! That is, if typechecker A doesn't do what the reference algorithm does, A is buggy -- not by virtue of hard thought, but simply by decree. (*) About "(presumed)": any number of languages, from Algol to Eiffel, have effective type-checking procedures (albeit that the fancier ones may, on occasion, delay a check to runtime, under well-defined circumstances). I expect you're using the word "check" where most of the world uses "inference" instead; e.g., your def f(): return f example is not a typechecking problem in my eyes (although it is an inference problem); def f() -> T: return f for some specific & explicit T would be what I call a typechecking problem. I'm not sure what Jeremy means. > ... > I agree completely with you when you say the type system > should be clearly stated. Ir would be hard to disagree with that one . > But the semantics for 'ill typed program' should be 'undefined'. > If you are silly enough to specify behaviour, then you are > allowing ill typed programs to have well defined meanings. > (which is absurd). Then I guess most languages on earth are "silly" wrt their type systems. granting-the-point-but-not-the-judgment-ly y'rs - tim From jeremy@cnri.reston.va.us Sat Jan 22 21:47:06 2000 From: jeremy@cnri.reston.va.us (Jeremy Hylton) Date: Sat, 22 Jan 2000 16:47:06 -0500 (EST) Subject: [Types-sig] how to do a typecase? (was: Static typing: Towards closure?) In-Reply-To: <3889BE8B.12A0618F@maxtal.com.au> References: <200001211633.LAA29205@eric.cnri.reston.va.us> <14472.38177.65905.81604@goon.cnri.reston.va.us> <3888C146.FCABD752@maxtal.com.au> <14472.54111.416650.255321@goon.cnri.reston.va.us> <3889BE8B.12A0618F@maxtal.com.au> Message-ID: <14474.9562.597719.952354@bitdiddle.cnri.reston.va.us> >>>>> "JS" == skaller writes: JS> So on the assumption that we want a rich enough type system to You're the only person I know who is making such an assumption. JS> describe Python, it isn't reasonable to require that 'if a JS> program checks out with one implementation and not with another, JS> then one of the type checkers has a bug' because it is JS> IMPOSSIBLE to construct a type checker that checks all cases. JS> IF you want a deterministic checker, it is necessary to weaken JS> the type system, making it less expressive (catching less errors JS> early). I have assumed that we are discussing a static type system that is decidably verifiable; there is a necessary sacrifice of expressiveness in order to allow static checking of programs. Your lengthy post refutes a strawman argument (although it's a lovely display of your erudition). Jeremy From tim_one@email.msn.com Sat Jan 22 23:48:25 2000 From: tim_one@email.msn.com (Tim Peters) Date: Sat, 22 Jan 2000 18:48:25 -0500 Subject: [Types-sig] lambdas, classes, tuples, default arguments, map In-Reply-To: <200001211934.OAA29597@eric.cnri.reston.va.us> Message-ID: <000801bf6533$2ce36220$132d153f@tim> [Tony Lownds] > I dont have much hope that we will be able to write the > true type of map using decl statements either. But I > think the builtin map could still be type checked, as > long as it is seen as a special case. I'm not too worried > about type checking a user's functions that have a similarly > hard signature, just map. [Guido] > Agreed. > # here is an approximation of map > decl map1 = def(def(T1)->R, [T1]) -> [R] > decl map2 = def(def(T1,T2)->R, [T1], [T2]) -> [R] > decl map3 = def(def(T1,T2,T3) -> R, \ > T1], [T2], [T3]) -> [R] > > decl map_signature = map1 & map2 & map3 > > decl map : map_signature > Very good -- you noticed that this requires the use > of '&', not '|', to combine the constituent types! Of > course. Something's confused here, or the meanings of "|" and "&" got swapped when I wasn't looking . decl intlist: [int] intlist = map(ord, ["a", "b", "c"]) conforms to map1 in the instantiation map1 = def(def(chr)->int, [chr]) -> [int] but it certainly doesn't conform to any of the other map_i (ord doesn't take 2, or 3, or ... arguments!). Since it conforms to (exactly) one of the map_i, it conforms to their union, not to their intersection. So somebody is missing *something* basic here. If it's me, educate me before I sin again . From skaller@maxtal.com.au Sun Jan 23 02:55:02 2000 From: skaller@maxtal.com.au (skaller) Date: Sun, 23 Jan 2000 13:55:02 +1100 Subject: [Types-sig] how to do a typecase? (was: Static typing: Towards closure?) References: <200001211633.LAA29205@eric.cnri.reston.va.us> <14472.38177.65905.81604@goon.cnri.reston.va.us> <3888C146.FCABD752@maxtal.com.au> <14472.54111.416650.255321@goon.cnri.reston.va.us> <3889BE8B.12A0618F@maxtal.com.au> <14474.9562.597719.952354@bitdiddle.cnri.reston.va.us> Message-ID: <388A6D86.E0A0E180@maxtal.com.au> Jeremy Hylton wrote: > > >>>>> "JS" == skaller writes: > > JS> So on the assumption that we want a rich enough type system to > > You're the only person I know who is making such an assumption. > I have assumed that we are discussing a static type system that is > decidably verifiable; there is a necessary sacrifice of expressiveness > in order to allow static checking of programs. Your lengthy post > refutes a strawman argument (although it's a lovely display of your > erudition). My assumption is that we want optional type checking and higher order functions. In general, such a system is likely to be undecidable. I also suspect that the proposed system, as it stands, is undecidable. Consider a function which accepts an X, or a tuple of X's, such functions do exist in the standard library I think: the declaration: Tuple -> Tuple | X -> Y #1 declares the argument type correctly -- assuming a strict left to right matching. However, this makes the | operator non-commutative. What that means, I'm not sure, but I guess it will make a mess of inference -- introducing control flow into the inference algorithm by way of ordering considerations. To give an example in y = f((1,) we want the X to be 'int', not Tuple(int). Note that this example in itself can NOT be declared inline with the syntax def f(x: Tuple | X) -> Tuple | Y because this is a _different_ type: Tuple | X -> Tuple | Y #2 [The first type is a subtype of this one. ie. stricter] So the problem, as I see it, is that the type system being desribed IS that rich. in the case above, SUPPOSE the client declared a function with an ARGUMENT like #1, and called it with an argument like #2, then what will the checker say? The program is CORRECT. I doubt that the checker can be required to compute that: this requires calculating the most general unifier, and for the rich type schema being described, it isn't decidable how to do that AFAIK. Just to be clear what I believe: if the type system supports higher order functions, it will be undecidable. I am NOT sure of this (but it sounds right). If it does not, map, apply and reduce cannot be declared. ** Once higher order functions are introduced, subtyping must be calculated, and it is indecidable for functions. ** Just to be sure you don't get confused here and think this consideration only applies to 'functional' functions, be aware it applies to 'functional objects' as well. In particular, it applies to class methods. For example: class X: def f(x:A) -> A: ... class Y(X): def f(x:B) -> C In order that class Y be a subtype of X***, it is necessary that B's f have C <= A <= B where <= means 'is subtype of'. This is the standard rule for methods -- returns are covariant, arguments are contravariant. Checking that requires signature matching on functions, that is, determining if a function signature is a subtype of another. *** Please note VERY carefully that subtyping is unrelated to subclassing. -- John (Max) Skaller, mailto:skaller@maxtal.com.au 10/1 Toxteth Rd Glebe NSW 2037 Australia voice: 61-2-9660-0850 homepage: http://www.maxtal.com.au/~skaller download: ftp://ftp.cs.usyd.edu/au/jskaller From scott@chronis.pobox.com Sun Jan 23 05:22:01 2000 From: scott@chronis.pobox.com (scott) Date: Sun, 23 Jan 2000 00:22:01 -0500 Subject: [Types-sig] lambdas, classes, tuples, default arguments, map In-Reply-To: <000801bf6533$2ce36220$132d153f@tim> References: <200001211934.OAA29597@eric.cnri.reston.va.us> <000801bf6533$2ce36220$132d153f@tim> Message-ID: <20000123002201.A92697@chronis.pobox.com> I think of the map example, as well as the signatures of the basic operators like '+' as a list of signatures, first match on the arguments determines the return type. That's how I see this example for map as well. That's also a lot like how ML spells it's polymorphic functions (and it's typecase-like thing). The only difference for me between this spelling of map's signature and a type of 'Int|None' is that the former is an ordered sequence to be searched and the latter is orderless, only one of the types may hold. I don't exactly see the relationship between '|' and '&' here though, just '|' and an ordered list. scott On Sat, Jan 22, 2000 at 06:48:25PM -0500, Tim Peters wrote: > [Tony Lownds] > > I dont have much hope that we will be able to write the > > true type of map using decl statements either. But I > > think the builtin map could still be type checked, as > > long as it is seen as a special case. I'm not too worried > > about type checking a user's functions that have a similarly > > hard signature, just map. > > [Guido] > > Agreed. > > > > # here is an approximation of map > > decl map1 = def(def(T1)->R, [T1]) -> [R] > > decl map2 = def(def(T1,T2)->R, [T1], [T2]) -> [R] > > decl map3 = def(def(T1,T2,T3) -> R, \ > > T1], [T2], [T3]) -> [R] > > > decl map_signature = map1 & map2 & map3 > > > decl map : map_signature > > > Very good -- you noticed that this requires the use > > of '&', not '|', to combine the constituent types! Of > > course. > > Something's confused here, or the meanings of "|" and "&" got swapped when I > wasn't looking . > > decl intlist: [int] > intlist = map(ord, ["a", "b", "c"]) > > conforms to map1 in the instantiation > > map1 = def(def(chr)->int, [chr]) -> [int] > > but it certainly doesn't conform to any of the other map_i (ord doesn't take > 2, or 3, or ... arguments!). Since it conforms to (exactly) one of the > map_i, it conforms to their union, not to their intersection. > > So somebody is missing *something* basic here. If it's me, educate me > before I sin again . > > > > _______________________________________________ > Types-SIG mailing list > Types-SIG@python.org > http://www.python.org/mailman/listinfo/types-sig From scott@chronis.pobox.com Sun Jan 23 16:26:08 2000 From: scott@chronis.pobox.com (scott) Date: Sun, 23 Jan 2000 11:26:08 -0500 Subject: [Types-sig] typecase hack In-Reply-To: <200001221541.KAA00535@eric.cnri.reston.va.us> References: <200001221541.KAA00535@eric.cnri.reston.va.us> Message-ID: <20000123112608.A95096@chronis.pobox.com> Interesting. Yes, it should work. Are you proposing this as the way to do it? It'd be nice if it works, but my vote has gotta side with the availability of a more elegant approach. I think this disambiguating thing is going to happen a lot, mostly as the result of things defaulting to None so often. I count 147 (is (not)?|==)\s*None matches in the standard library. Neat, though. scott On Sat, Jan 22, 2000 at 10:41:16AM -0500, Guido van Rossum wrote: > Just asked me to explain the need for typecase again, and I realized > that there's a parallel with except clauses. With only a little bit > of imagination we could make the official way to spell a typecase as > follows: > > try: raise x > except int, xi: ...use xi: int... > except str, xs: ...use xs: str... > > I'm not saying this is particularly elegant -- but it should work, > once types and classes are more similar. > > --Guido van Rossum (home page: http://www.python.org/~guido/) > > _______________________________________________ > Types-SIG mailing list > Types-SIG@python.org > http://www.python.org/mailman/listinfo/types-sig From tony@metanet.com Sun Jan 23 19:13:38 2000 From: tony@metanet.com (Tony Lownds) Date: Sun, 23 Jan 2000 11:13:38 -0800 (PST) Subject: [Types-sig] lambdas, classes, tuples, default arguments, map In-Reply-To: <000801bf6533$2ce36220$132d153f@tim> Message-ID: > Something's confused here, or the meanings of "|" and "&" got swapped when I > wasn't looking . The decl is saying that the same map function can conform to map1 *and* map2 *and* map3. If it said map1 *or* map2 etc, then the programmer would have to check to make sure the current value of the map variable is the alternative that can handle the way its about to be called. > decl intlist: [int] > intlist = map(ord, ["a", "b", "c"]) > > conforms to map1 in the instantiation > > map1 = def(def(chr)->int, [chr]) -> [int] > > but it certainly doesn't conform to any of the other map_i (ord doesn't take > 2, or 3, or ... arguments!). Yes it does! Assuming the it you are referring to is the map function, not some instantiation or something else. You can call that same function, the same func_object, as map2 or map3 specifies. > Since it conforms to (exactly) one of the > map_i, it conforms to their union, not to their intersection. No, it conforms to all at the same time. I wasn't using '&' as a intersection operator but as a combination operator. -Tony From jeremy@cnri.reston.va.us Sun Jan 23 19:50:33 2000 From: jeremy@cnri.reston.va.us (Jeremy Hylton) Date: Sun, 23 Jan 2000 14:50:33 -0500 (EST) Subject: [Types-sig] how to do a typecase? (was: Static typing: Towards closure?) In-Reply-To: <388A6D86.E0A0E180@maxtal.com.au> References: <200001211633.LAA29205@eric.cnri.reston.va.us> <14472.38177.65905.81604@goon.cnri.reston.va.us> <3888C146.FCABD752@maxtal.com.au> <14472.54111.416650.255321@goon.cnri.reston.va.us> <3889BE8B.12A0618F@maxtal.com.au> <14474.9562.597719.952354@bitdiddle.cnri.reston.va.us> <388A6D86.E0A0E180@maxtal.com.au> Message-ID: <14475.23433.309254.578265@bitdiddle.cnri.reston.va.us> >>>>> "JS" == skaller writes: JS> My assumption is that we want optional type checking and JS> higher order functions. In general, such a system is likely to JS> be undecidable. I also suspect that the proposed system, as it JS> stands, is undecidable. JS> Consider a function which accepts an X, or a tuple of X's, JS> such functions do exist in the standard library I think: the JS> declaration: I don't understand the problem you're describing. If we have a function with the type f: (X,) -> (Y,) | X -> Y (using Guido's notation for tuple) and we call the function with an argument of type (1,) f((1,)) then this function application should check. Since (X,) -> (Y,) <: X -> Y it seems like the first half is unnecessary. I imagine you're trying to specify a type that restricts the function to only returning a Y-tuple if it's argument is an X-tuple, but also accepts other non-tuple types. I don't think it will be possible to specify a type that enforces that restriction (although I'm not sure). Did I understand this example correctly? JS> Just to be clear what I believe: if the type system supports JS> higher order functions, it will be undecidable. I am NOT sure JS> of this (but it sounds right). I don't understand this claim. I am familiar with many languages that have both sound static type systems and higher order functions. Am I mistaking your here? Perhaps you mean that a language with higher order funtions and , is undecidable? If so, exactly what property is it that gets in the way? Jeremy From jeremy@cnri.reston.va.us Sun Jan 23 19:58:20 2000 From: jeremy@cnri.reston.va.us (Jeremy Hylton) Date: Sun, 23 Jan 2000 14:58:20 -0500 (EST) Subject: [Types-sig] how to do a typecase? (was: Static typing: Towards closure?) In-Reply-To: <000101bf651d$f21850c0$132d153f@tim> References: <3889BE8B.12A0618F@maxtal.com.au> <000101bf651d$f21850c0$132d153f@tim> Message-ID: <14475.23900.397638.971158@bitdiddle.cnri.reston.va.us> >>>>> "TP" == Tim Peters writes: TP> Proving the *equivalence* of two distinct typecheckers is TP> nevertheless not precluded by the (presumed(*)) unsolvability of TP> the typechecking problem. [...] TP> (*) About "(presumed)": any number of languages, from Algol to TP> Eiffel, have effective type-checking procedures (albeit that the TP> fancier ones may, on occasion, delay a check to runtime, under TP> well-defined circumstances). I expect you're using the word TP> "check" where most of the world uses "inference" instead; e.g., TP> your TP> def f(): return f TP> example is not a typechecking problem in my eyes (although it is TP> an inference problem); TP> def f() -> T: return f TP> for some specific & explicit T would be what I call a TP> typechecking problem. I'm not sure what Jeremy means. I didn't mean to suggest an inference problem. I don't see how we can specify a type for the function either, but I'm not sure that it's much of a loss. Who cares if we can't typecheck programs with functions like "def f(): return f". I tried to write the same function in ocaml and got nowhere: ># let rec f () = f;; >This expression has type unit -> 'a but is here used with type 'a This response seems sensible to me. Jeremy From tim_one@email.msn.com Sun Jan 23 20:11:49 2000 From: tim_one@email.msn.com (Tim Peters) Date: Sun, 23 Jan 2000 15:11:49 -0500 Subject: [Types-sig] how to do a typecase? (was: Static typing: Towards closure?) In-Reply-To: <388A6D86.E0A0E180@maxtal.com.au> Message-ID: <000001bf65de$15a5dd80$022d153f@tim> [John Skaller] > My assumption is that we want optional type checking > and higher order functions. I think everyone agrees on that, yes. Although for "optional type checking" I am *not* reading "comprehensive type inference in the absence of explicit declarations", but "optional explicit declarations and no guarantees of type safety in their absence". > In general, such a system is likely to be undecidable. I'd amend that to add "without extreme care". Strongly typed languages like Haskell and Miranda support both sophisticated polymorphism and higher-order functions, without requiring declarations, yet have provably sound & effective type inference (in the technical senses of those words: sound == "correct", effective == "always terminates"). This wasn't easy to achieve, though, and a mountain of work went into refining the base Hindley-Milner type system (which was fully developed *before* e.g. Haskell built on top of it). Some subtle technical restrictions were required. http://www.haskell.org/tutorial/pitfalls.html is a good starting-off point for exploring this topic (the "pitfalls" relate entirely to the type system). > I also suspect that the proposed system, as it stands, > is undecidable. For most plausible ways of filling in the holes, I believe that's correct, except that ... Guido is (maybe too ) fond of saying that Python is "a 95% language", and I expect that will be a Guiding Principle here. That is, the type system will be fudged in ways such that it remains tractable, by some combination of compromise, restriction, and plain giving up. For example, it seems already to have been agreed that the type system will not be able to express the type of Python's "map" function (although there's hope that the type checker will "know it anyway", as a special case). This approach doesn't appeal to the axiomatic-deductive mind, so expect not to like it. The technical issues are so intricate, though, that our little group of pragmatic part-timers likely isn't capable of doing significantly better than that. Hindley-Milner is about as expressive as "provably correct" gets even today. [some great examples of difficulties, which I didn't understand at first because people have been writing examples such that "|" binds tighter than "->"; so e.g. I first read John's Tuple -> Tuple | X -> Y as Tuple -> (Tuple | X) -> Y instead of the intended (from context) (Tuple -> Tuple) | (X -> Y) ] > ... > The program is CORRECT. I doubt that the checker > can be required to compute that: this requires calculating > the most general unifier, and for the rich type schema > being described, it isn't decidable how to do that AFAIK. Even Hindley-Milner gives up at times (although they don't phrase it in such raw terms), in the sense of forbidding programs that may nevertheless be provably correct by going outside the H-M approach (the Haskell "pitfall" 12.1 (restriction to "let-bound polymorphism") is a prime example of this). > Just to be clear what I believe: if the type system > supports higher order functions, it will be undecidable. > I am NOT sure of this (but it sounds right). As above, HOFs alone aren't enough to break the camel's back, but they sure do put a load it. Alas, I don't know of a simple characterization of what is (in aggregate) enough to break it, and the H-M restrictions are so technical in nature as to suggest a simple characterization doesn't exist. But I'm no expert in this stuff either. From skaller@maxtal.com.au Sun Jan 23 21:53:11 2000 From: skaller@maxtal.com.au (skaller) Date: Mon, 24 Jan 2000 08:53:11 +1100 Subject: [Types-sig] how to do a typecase? (was: Static typing: Towards closure?) References: <000001bf65de$15a5dd80$022d153f@tim> Message-ID: <388B7847.9229FE72@maxtal.com.au> Tim Peters wrote: > > [John Skaller] > > My assumption is that we want optional type checking > > and higher order functions. > > I think everyone agrees on that, yes. Although for "optional type checking" > I am *not* reading "comprehensive type inference in the absence of explicit > declarations", but "optional explicit declarations and no guarantees of type > safety in their absence". Agreed. The way I saw it was that type declarations could be added as desired, to improve diagnostics and optimisation opportunities. This would mean that by default, no assurance of type correctness is obtained, at least until run time (and maybe not even then). OTOH, a separate type checking tool could report IF in fact the code contains enough information to provide an assurance of type correctness (and maximal optimisation opportunities) allowing the programmer to add extra declarations if desired. > > In general, such a system is likely to be undecidable. > > I'd amend that to add "without extreme care". Strongly typed languages like > Haskell and Miranda support both sophisticated polymorphism and higher-order > functions, without requiring declarations, yet have provably sound & > effective type inference (in the technical senses of those words: sound == > "correct", effective == "always terminates"). I can't say about Haskell or Miranda .. but Ocaml's type checker can go into an infinite loop, and it is NOT expressive enough to handle some quite common constructions. Perhaps this is because it supports imperative programming, separate compilation, higher order functors, constraints, and object orientation, whereas Haskell at least is purely functional. > This wasn't easy to achieve, though, and a mountain of work went into > refining the base Hindley-Milner type system (which was fully developed > *before* e.g. Haskell built on top of it). Some subtle technical > restrictions were required. Yeah, I can imagine. > > I also suspect that the proposed system, as it stands, > > is undecidable. > > For most plausible ways of filling in the holes, I believe that's correct, > except that ... > > Guido is (maybe too ) fond of saying that Python is "a 95% language", > and I expect that will be a Guiding Principle here. That is, the type > system will be fudged in ways such that it remains tractable, by some > combination of compromise, restriction, and plain giving up. I think that this is my point. That is: I think this is reasonable (it is what ocaml does). But people seem to think that there will be a certain guarrantee that a module can be strictly type checked. On the other hand, I think that it makes sense to require diagnostics where a (minimal) reference implementation produces them, and require acceptance where a (minimal) reference implementation can prove correctness, and require nothing at all in the other cases. This means, in particular, that some checkers will accept more programs than others -- including the reference implementation -- and some will reject more than others -- including the reference implementation -- and the only time you can say the checker has a bug is if a case which the reference implementation accepts is not accepted, and the case which the reference implementation rejects is not rejected, it is never a bug to fail to accept a correct program if the reference implementation doesn't accept it, nor to fail to reject a wrong program if the reference implementation doesn't reject it. This leaves plenty of scope for improving checkers, and plenty of scope for fully optimised programs to core dump if they optimise on the _assumption_ that the program obeys the type assertions implied by declarations. It also means highly optimised programs can be produced without risking an accusation that the optimising compiler is non-compliant. > For example, > it seems already to have been agreed that the type system will not be able > to express the type of Python's "map" function (although there's hope that > the type checker will "know it anyway", as a special case). Yes, it is the same in C and C++ and most other languages: the compiler not only 'knows' the type of built-in library functions, it ALSO 'knows' the semantics, and can generate highly optimised code in these cases by utilising this semantic information. > [some great examples of difficulties, which I didn't understand > at first because people have been writing examples such that > "|" binds tighter than "->"; so e.g. I first read John's > > Tuple -> Tuple | X -> Y > > as > > Tuple -> (Tuple | X) -> Y > > instead of the intended (from context) > > (Tuple -> Tuple) | (X -> Y) > ] Ugg, sorry, I was sloppy here! > > The program is CORRECT. I doubt that the checker > > can be required to compute that: this requires calculating > > the most general unifier, and for the rich type schema > > being described, it isn't decidable how to do that AFAIK. > > Even Hindley-Milner gives up at times (although they don't phrase it in such > raw terms), in the sense of forbidding programs that may nevertheless be > provably correct by going outside the H-M approach (the Haskell "pitfall" > 12.1 (restriction to "let-bound polymorphism") is a prime example of this). Ah, but the point is that at least it either gives up noisily, or terminates and assures one of correctness. This is not easy to achieve. > > Just to be clear what I believe: if the type system > > supports higher order functions, it will be undecidable. > > I am NOT sure of this (but it sounds right). > > As above, HOFs alone aren't enough to break the camel's back, but they sure > do put a load it. Alas, I don't know of a simple characterization of what > is (in aggregate) enough to break it, and the H-M restrictions are so > technical in nature as to suggest a simple characterization doesn't exist. > But I'm no expert in this stuff either. For Python, I think we can guarrantee that _straight line_ code is always checked, provided all the symbols are typed. In other words, if we know the types of identifiers and functions, then the correctness of all expressions and assignments can be checked. This assurance can be extended to loops, but we have to give up when we reach a try/except block. Certain builin functions cannot be typed properly, though, such as things that call map, exec, eval, etc. In my opinion, this will be enough to gain _massive_ optimisation opportunities: it is mainly small inner loops that need to be optimised anyhow: many of these will be quite simple, and simply typed. I also expect considerable benefit in terms of both diagnostics and possibly readability for general use -- but without a _guarrantee_ of type correctness which some other people seem to be striving for. It is a good aim, but I think it is unrealisable. PS: I really don't like the 'decl' keyword, at least for code (as opposed to separate interface files). Nor do I like the 'type' mini-language idea much. The way I see it, this is a flaw in conventional type systems. A more 'Pythonic' technique would KEEP the idea that types were run time objects (as classes are), but still allow significant optimisation and early diagnosis IF enough information were provided and the usage were simple enough. I STILL think that this can best be achieved by conservative syntax extensions like Greg Stein's type assertion operator, plus some restrictions on required behaviour. This isn't enough for best quality checking -- but otherwise the static typing language is likely to be MUCH more complex than Python itself. OTOH, I agree with Guido that parameterised types are useful -- but I suggest that these can be provided by changing the run-time type system (Vyper provides parameterised types now, although the mechanism is a bit crude as yet). The requisite type assertion operator can be implemented in a few minutes. Changing the CPython type system to allow any object as a type should also be relatively easy [objects of type TypeType are still allowed, and are treated specially] This will bring CPython up to the current status of Vyper. It will allow Stein to get on with writing the type checker, and others to get on with optimising C code generators, and it will allow programmers to begin annotating their code right away. What Python needs -- desparately -- is a way of building categorical sums (variants) like ML's type t = X | Y of int Switching on 'type' as in a typecase is a BAD idea. This is done in C++ using if(X *x=dynamic_cast(y)) { ...} that is, using a small modification to the C 'if' construction, Stroustrup refused to bless this hackery with a special new construction. Because it is the WRONG idea, switching on type. This can be seen by considering the construction: type measurement = Metres of float | Feet of float the values have the SAME type. The switching must occur on an explicit tag. Interestingly, there are TWO constructions already in the Grammar that allow this: (tag, value) # a tuple tag: value # dictionary entry .. apart from using classes (ugly***): class Tag: def __init_(self, value): self.value = value I've actually implemented: { 1 : 10, 2 : 20 } { 1: lambda x: x 2: lambda x: x * x 3: lambda x: x * x * x } which is a kind of switch allowing multiple selections. The result is { 1: 10, 2: 400 }. Unfortunately, the parse is 'atom with trailers' and the trailer must use explicit {}. *** this is wrong, because the type of 'Metres 20.0' is the same as the type of 'Feet 20.0', namely 'measurement'. Using classes confuses tagging (unification), with subtyping, which are diametrically opposed in nature. [Unification is a disjoint union, subtyping is a subset relation] This is one of the most serious faults in most OO systems, and the most serious abuse of inheritance. Luckily, advanced systems like ocaml get all this right (variants are independent of classes which allow inheritance, which is independent of signature based subtyping) -- even if the result is somewhat clumbsily written, and has annoying restrictions. ------------------ In any case, I think the best thing for Python is the minimal syntactic change and architectural change -- because anything else will ALSO be incomplete/inadequately expressive/non-deterministic, and in ADDITION it will likely be very complex. Here's a possible scenario: 1) Allow any object to be a type object 2) Provide operator ! for expressions and function parameters (and returns?) 3) Disallow certain operations like writing to module attributes after the module is imported 4) Unify classes/type better I believe these features will significantly improve opportunities for early error diagnosis and optimisation --WITHOUT adding any complex new sublanguages or constructions. The result will be a 95% improvement in static assurance :-) Perhaps, with this done, we can learn enough to make the next improvement. In some sense, I like Greg's approach: you implement SOMETHING minimal, then try to improve it, until you find the difference between 'tricky to get right' and 'there is not enough information to go further -- we need to know THIS'. When we know what THIS really is, we can rethink how to provide that information in the language, so as to improve the performance of working checkers. The result will never be as good as a 'designed from scratch using mathematics' language, but if you want that, you'd be using Charity. -- John (Max) Skaller, mailto:skaller@maxtal.com.au 10/1 Toxteth Rd Glebe NSW 2037 Australia voice: 61-2-9660-0850 homepage: http://www.maxtal.com.au/~skaller download: ftp://ftp.cs.usyd.edu/au/jskaller From skaller@maxtal.com.au Sun Jan 23 22:14:04 2000 From: skaller@maxtal.com.au (skaller) Date: Mon, 24 Jan 2000 09:14:04 +1100 Subject: [Types-sig] how to do a typecase? (was: Static typing: Towards closure?) References: <200001211633.LAA29205@eric.cnri.reston.va.us> <14472.38177.65905.81604@goon.cnri.reston.va.us> <3888C146.FCABD752@maxtal.com.au> <14472.54111.416650.255321@goon.cnri.reston.va.us> <3889BE8B.12A0618F@maxtal.com.au> <14474.9562.597719.952354@bitdiddle.cnri.reston.va.us> <388A6D86.E0A0E180@maxtal.com.au> <14475.23433.309254.578265@bitdiddle.cnri.reston.va.us> Message-ID: <388B7D2C.3C08F2EA@maxtal.com.au> Jeremy Hylton wrote: > JS> Consider a function which accepts an X, or a tuple of X's, > JS> such functions do exist in the standard library I think: the > JS> declaration: > > I don't understand the problem you're describing. If we have a > function with the type > f: (X,) -> (Y,) | X -> Y (using Guido's notation for tuple) > and we call the function with an argument of type (1,) > f((1,)) > then this function application should check. > I imagine you're trying to specify a type that restricts the function > to only returning a Y-tuple if it's argument is an X-tuple, but also > accepts other non-tuple types. I don't think it will be possible to > specify a type that enforces that restriction (although I'm not sure). > > Did I understand this example correctly? Yes, I think so. Some functions accept a NON-Tuple X as a singleton tuple, where a tuple is required 'in principal'. Such 'conveniences' are always a bad idea -- but you can't tell amateur language designers that kind of thing. They will always come up with an argument that it is sound in the current system -- and fail to see that it is also fragile and will break if the system is generalised. This is the main reason C++ is such a mess: it is based on C, which is one of the worst designed languages around. > JS> Just to be clear what I believe: if the type system supports > JS> higher order functions, it will be undecidable. I am NOT sure > JS> of this (but it sounds right). > > I don't understand this claim. I am familiar with many languages that > have both sound static type systems and higher order functions. Am I > mistaking your here? # let rec f () = f;; This expression has type unit -> 'a but is here used with type 'a Ocaml will not accept this function. Python does: def f(): return f You can't do this in C or C++ either. The type involves type recursion. But the function is OK: it is the type system which is broken here. -- John (Max) Skaller, mailto:skaller@maxtal.com.au 10/1 Toxteth Rd Glebe NSW 2037 Australia voice: 61-2-9660-0850 homepage: http://www.maxtal.com.au/~skaller download: ftp://ftp.cs.usyd.edu/au/jskaller From skaller@maxtal.com.au Sun Jan 23 22:25:59 2000 From: skaller@maxtal.com.au (skaller) Date: Mon, 24 Jan 2000 09:25:59 +1100 Subject: [Types-sig] how to do a typecase? (was: Static typing: Towards closure?) References: <3889BE8B.12A0618F@maxtal.com.au> <000101bf651d$f21850c0$132d153f@tim> <14475.23900.397638.971158@bitdiddle.cnri.reston.va.us> Message-ID: <388B7FF7.5DBB6773@maxtal.com.au> Jeremy Hylton wrote: > TP> def f(): return f > I don't see how we can specify a type for the function either, but I'm > not sure that it's much of a loss. Who cares if we can't typecheck > programs with functions like "def f(): return f". The point is that this function is the _simplest_ example of a wide class of functions which will have the same problem. The question is: what will a type checker do, seeing this function? Will it loop forever trying to compute the type of f? The answer is: probably. And we must allow this if type declations are optional. The only way to disallow the above function is to require everything be explicitly typed, in which case there is no way to name the type of the function above. In THIS simple case it is easy to see an infinite recursion is involved and just type the function as () -> Any () -> () -> Any () -> () -> () -> Any .. etc [each of the later declarations is 'better' than any previous ones] but it isn't clear how to detect this in general -- without eliminating useful (tractable) use of type recursion such as: type node = Node of node | Int of int in fact, it is possible to allow 'f' above to be type by introducing a type 'fix' operator. -- John (Max) Skaller, mailto:skaller@maxtal.com.au 10/1 Toxteth Rd Glebe NSW 2037 Australia voice: 61-2-9660-0850 homepage: http://www.maxtal.com.au/~skaller download: ftp://ftp.cs.usyd.edu/au/jskaller From paul@prescod.net Sun Jan 23 22:49:16 2000 From: paul@prescod.net (Paul Prescod) Date: Sun, 23 Jan 2000 16:49:16 -0600 Subject: [Types-sig] Syntax issues References: <200001191722.MAA19286@eric.cnri.reston.va.us> Message-ID: <388B856C.9F1C9C54@prescod.net> Guido van Rossum wrote: > > .... > > > Paul suggests something similar, but uses a tuple of two strings. I > don’t see the point of that (besides, such a tuple ends up being > evaluated and then thrown away at run time; a simple string is thrown > away during code generation). I want to avoid confusion with docstrings in contexts where either would be legal. > There's one more idea that I want to discuss: once the in-line syntax > is accepted, the 'decl' keyword may be redundant (in most cases > anyway). I think that it is a useful indicator about what is evaluated at compile time, as Python's (otherwise redundant) colon is a useful indicator that a block is about to start. -- Paul Prescod - ISOGEN Consulting Engineer speaking for himself Earth will soon support only survivor species -- dandelions, roaches, lizards, thistles, crows, rats. Not to mention 10 billion humans. - Planet of the Weeds, Harper's Magazine, October 1998 From paul@prescod.net Sun Jan 23 22:49:26 2000 From: paul@prescod.net (Paul Prescod) Date: Sun, 23 Jan 2000 16:49:26 -0600 Subject: [Types-sig] Type-checked code calling unchecked code References: <200001191722.MAA19286@eric.cnri.reston.va.us> Message-ID: <388B8576.618B4AAD@prescod.net> Guido van Rossum wrote: > > ... > > When a checked module imports an unchecked module, all objects in the > unchecked module (and the unchecked module itself) are assumed to be of > type 'any', and the checked module is typechecked accordingly. I think that there is a problem here. j = uncheckedmodule.j j() Do I really have to declare that j is a callable to allow this to be type-safe? Shouldn't we sniff out "obvious" type information (perhaps functions versus classes versus everything else?). -- Paul Prescod - ISOGEN Consulting Engineer speaking for himself Earth will soon support only survivor species -- dandelions, roaches, lizards, thistles, crows, rats. Not to mention 10 billion humans. - Planet of the Weeds, Harper's Magazine, October 1998 From guido@CNRI.Reston.VA.US Mon Jan 24 01:33:59 2000 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Sun, 23 Jan 2000 20:33:59 -0500 Subject: [Types-sig] Type-checked code calling unchecked code In-Reply-To: Your message of "Sun, 23 Jan 2000 16:49:26 CST." <388B8576.618B4AAD@prescod.net> References: <200001191722.MAA19286@eric.cnri.reston.va.us> <388B8576.618B4AAD@prescod.net> Message-ID: <200001240133.UAA01094@eric.cnri.reston.va.us> > Guido van Rossum wrote: > > > > When a checked module imports an unchecked module, all objects in the > > unchecked module (and the unchecked module itself) are assumed to be of > > type 'any', and the checked module is typechecked accordingly. [Paul Prescod] > > I think that there is a problem here. > > j = uncheckedmodule.j > > j() > > Do I really have to declare that j is a callable to allow this to be > type-safe? Shouldn't we sniff out "obvious" type information (perhaps > functions versus classes versus everything else?). Of course not -- 'any' is callable. However, if you wrote decl i: int i = j() You'd get a type error because the return type is any. With Greg, you could write i = j() ! int or with typecase, you could write typecase j(): int, ji: i = i else: ... --Guido van Rossum (home page: http://www.python.org/~guido/) From tim_one@email.msn.com Mon Jan 24 03:55:20 2000 From: tim_one@email.msn.com (Tim Peters) Date: Sun, 23 Jan 2000 22:55:20 -0500 Subject: [Types-sig] how to do a typecase? (was: Static typing: Towards closure?) In-Reply-To: <388B7FF7.5DBB6773@maxtal.com.au> Message-ID: <000201bf661e$d57f3940$902d153f@tim> [John Skaller, on def f(): return f] > ... > The question is: what will a type checker do, > seeing this function? Will it loop forever trying to compute > the type of f? I still won't call this a "type checking" problem; it's a (potential) problem for a type *inferencer*, but: > The answer is: probably. And we must allow this if type > declations are optional. The only way to disallow the above > function is to require everything be explicitly typed, in > which case there is no way to name the type of the function > above. Bingo. Checking that use matches declaration is easier than inferring declaration from use. "Mere checking" is hard enough for a first version, though! From jeremy@cnri.reston.va.us Mon Jan 24 04:14:06 2000 From: jeremy@cnri.reston.va.us (Jeremy Hylton) Date: Sun, 23 Jan 2000 23:14:06 -0500 (EST) Subject: [Types-sig] how to do a typecase? (was: Static typing: Towards closure?) In-Reply-To: <388B7FF7.5DBB6773@maxtal.com.au> References: <3889BE8B.12A0618F@maxtal.com.au> <000101bf651d$f21850c0$132d153f@tim> <14475.23900.397638.971158@bitdiddle.cnri.reston.va.us> <388B7FF7.5DBB6773@maxtal.com.au> Message-ID: <14475.53646.954417.163189@bitdiddle.cnri.reston.va.us> >>>>> "JS" == skaller writes: [On the subject of "def f(): return f"] [I wrote:] >> I don't see how we can specify a type for the function either, >> but I'm not sure that it's much of a loss. Who cares if we can't >> typecheck programs with functions like "def f(): return f". JS> The point is that this function is the _simplest_ example of a JS> wide class of functions which will have the same problem. [...] JS> previous ones] but it isn't clear how to detect this in general JS> -- without eliminating useful (tractable) use of type recursion JS> such as: JS> type node = Node of node | Int of int It isn't clear to me either, but I see that in Ocaml I can successfully evaluate: "type node = Node of node | Int of int" but not "let rec f () = f". This suggests to me that it is possible to allow recursive types without having a solution to latter problem. Jeremy From tim_one@email.msn.com Mon Jan 24 04:46:00 2000 From: tim_one@email.msn.com (Tim Peters) Date: Sun, 23 Jan 2000 23:46:00 -0500 Subject: [Types-sig] lambdas, classes, tuples, default arguments, map In-Reply-To: Message-ID: <000701bf6625$e9206260$902d153f@tim> [Tim] > Something's confused here, or the meanings of "|" and "&" got > swapped when I wasn't looking . [Tony Lownds] > The decl is saying that the same map function can conform > to map1 *and* map2 *and* map3. If it said map1 *or* map2 etc, > then the programmer would have to check to make sure the > current value of the map variable is the alternative that > can handle the way its about to be called. Let's make this more formal, so we stop stumbling over words. My understanding has been that, in the scope of decl var: T1 | T2 each static (textual) reference to var must have at least one of isinstance(var, T1) isinstance(var, T2) be true at runtime. But in the scope of decl var: T1 & T2 each static reference to var must have *both* of isinstance(var, T1) isinstance(var, T2) be true at runtime. The latter follows from Guido's "motivating example" of multiple interfaces in Java; to say that something is "Readable & Writable" is not to say that something *can* be Readable and (or) Writable, it's to say that the thing must be both at the same time. Having spelled that out, it is true that map satisfies all of isinstance(map, map1) isinstance(map, map2) ... at each static site. Hmm -- so I have to admit "&" is appropriate after all. Thanks! > ... > I wasn't using '&' as a intersection operator but as a > combination operator. Same thing -- if we take the set of all functions that conform to map1, and the set of all that conform to map2, and ... map is in the intersection of those sets. That's (roughly, but not too roughly) what I meant by "intersection" and "union". educated!-ly y'rs - tim From scott@chronis.pobox.com Mon Jan 24 05:57:06 2000 From: scott@chronis.pobox.com (scott) Date: Mon, 24 Jan 2000 00:57:06 -0500 Subject: [Types-sig] Type-checked code calling unchecked code In-Reply-To: <200001240133.UAA01094@eric.cnri.reston.va.us> References: <200001191722.MAA19286@eric.cnri.reston.va.us> <388B8576.618B4AAD@prescod.net> <200001240133.UAA01094@eric.cnri.reston.va.us> Message-ID: <20000124005706.A98461@chronis.pobox.com> On Sun, Jan 23, 2000 at 08:33:59PM -0500, Guido van Rossum wrote: > > Guido van Rossum wrote: > > > > > > When a checked module imports an unchecked module, all objects in the > > > unchecked module (and the unchecked module itself) are assumed to be of > > > type 'any', and the checked module is typechecked accordingly. > > [Paul Prescod] > > > > I think that there is a problem here. > > > > j = uncheckedmodule.j > > > > j() > > > > Do I really have to declare that j is a callable to allow this to be > > type-safe? Shouldn't we sniff out "obvious" type information (perhaps > > functions versus classes versus everything else?). > > Of course not -- 'any' is callable. However, if you wrote > > decl i: int > i = j() > > You'd get a type error because the return type is any. > > With Greg, you could write > > i = j() ! int > Just wanted to point out that in the case of '!', you get a runtime error if you are wrong. in the case of typecase, you get a compile time error if it's wrong (typewise, of course). I feel like I'm repeating myself re: the runtime errors associated with !, but I think it's important that it be pointed out, since the goal as I understand it is greater compile time safety, and since the issue hasn't really been addressed at all. scott > or with typecase, you could write > > typecase j(): > int, ji: i = i > else: ... > > --Guido van Rossum (home page: http://www.python.org/~guido/) > > _______________________________________________ > Types-SIG mailing list > Types-SIG@python.org > http://www.python.org/mailman/listinfo/types-sig From scott@chronis.pobox.com Mon Jan 24 06:22:58 2000 From: scott@chronis.pobox.com (scott) Date: Mon, 24 Jan 2000 01:22:58 -0500 Subject: [Types-sig] how to do a typecase? (was: Static typing: Towards closure?) In-Reply-To: <388B7847.9229FE72@maxtal.com.au> References: <000001bf65de$15a5dd80$022d153f@tim> <388B7847.9229FE72@maxtal.com.au> Message-ID: <20000124012258.B98461@chronis.pobox.com> [...] > > Here's a possible scenario: > > 1) Allow any object to be a type object > 2) Provide operator ! for expressions and function > parameters (and returns?) > 3) Disallow certain operations like writing to module attributes > after the module is imported > 4) Unify classes/type better > > I believe these features will significantly improve opportunities > for early error diagnosis and optimisation --WITHOUT adding any > complex new sublanguages or constructions. > > The result will be a 95% improvement in static assurance :-) > > Perhaps, with this done, we can learn enough to make the next > improvement. In some sense, I like Greg's approach: you implement > SOMETHING minimal, then try to improve it, until you find > the difference between 'tricky to get right' and 'there is not > enough information to go further -- we need to know THIS'. > The idea of a scaled down system is certainly worth serious consideration. I think it's important to give a decently flexible system a try before reverting to something more scaled down, though. Anyway, as far as scaled down systems go, the one that makes most sense to me is: 1) disallow OR'd types. 2) consider None as NULL in C -- any type can be None legally (as any pointer can in C) 3) introduce declarations of compound types much like we have discussed. 4) drop ! 5) drop typecase 6) consider all code that requires OR to be un type checkable This would allow a good bit of code to be subject to type checks in a very simple system (as far as type checkers go). It would also afford the most optimization down the road since it would not require any additional runtime information to be managed by python. Additionally, since it would really only allow declarations of existing types, it would allow growth in the area of user defined types (and the correspanding runtime changes) as a separate system which could be developed over the long period of time that it deserves. I'm not there yet, but this sort of thing becomes quite tempting at times. scott ------ example code from above system ------ decl x: string decl y: int decl d: {string: int} import string, sys d = {} for y in range(26): d[string.lowercase[y]] = y # OK, any type can be None, and None isn't even part of the # signature of .get anyway y = d.get(sys.argv[0][-1]) y = "foo" # ERROR From tim_one@email.msn.com Mon Jan 24 07:13:13 2000 From: tim_one@email.msn.com (Tim Peters) Date: Mon, 24 Jan 2000 02:13:13 -0500 Subject: [Types-sig] how to do a typecase? (was: Static typing: Towards closure?) In-Reply-To: <14475.53646.954417.163189@bitdiddle.cnri.reston.va.us> Message-ID: <000001bf663a$7ad551c0$022d153f@tim> [Jeremy Hylton] > It isn't clear to me either, but I see that in Ocaml I can > successfully evaluate: > "type node = Node of node | Int of int" > but not > "let rec f () = f". > This suggests to me that it is possible to allow recursive types > without having a solution to latter problem. FYI, the function example is a non-starter in Haskell, because Haskell has no 0-argument functions (in a purely functional language, a 0-argument function can never return different things on different calls, so they generally don't allow for that -- you define a constant instead). Here's a related Haskell example: "f x = f". In ocaml (I haven't used that) you'd probably write something like "let rec f x = f;;" instead. Type checking ERROR "e:\hugs\ex.hs" (line 1): Type error in function binding *** term : f *** type : b -> a *** does not match : a *** because : unification would give infinite type "The rules" are spelled out in the Haskell reference. I downloaded the ocaml reference, and after failing to figure out *its* rules, bumped into this paragraph: No attempt has been made at mathematical rigor: words are employed with their intuitive meaning, without further definition. As a consequence, the typing rules have been left out, by lack of the mathematical framework required to express them, while they are definitely part of a full formal definition of the language. That's a real help . From skaller@maxtal.com.au Mon Jan 24 13:24:23 2000 From: skaller@maxtal.com.au (skaller) Date: Tue, 25 Jan 2000 00:24:23 +1100 Subject: [Types-sig] Type-checked code calling unchecked code References: <200001191722.MAA19286@eric.cnri.reston.va.us> <388B8576.618B4AAD@prescod.net> <200001240133.UAA01094@eric.cnri.reston.va.us> <20000124005706.A98461@chronis.pobox.com> Message-ID: <388C5287.D7C30D52@maxtal.com.au> scott wrote: > > With Greg, you could write > > > > i = j() ! int > > > > Just wanted to point out that in the case of '!', you get a runtime > error if you are wrong. in the case of typecase, you get a compile > time error if it's wrong (typewise, of course). I feel like I'm > repeating myself re: the runtime errors associated with !, but I think > it's important that it be pointed out, since the goal as I understand > it is greater compile time safety, and since the issue hasn't really > been addressed at all. Greg Stein's semantics are NOT the ones I advocate -- but the syntax is fine. I define x ! y as an assertion, and like all assertions in Vyper -- and indeed in Python -- if they are wrong, then the language translator is NOT required to raise a TypeError, in fact, it can do anything it wants. In particular, it can ASSUME that the assertion holds, and optimise accordingly, and it can PROVE (perhaps) that the assertion is false, and issue a compile time diagnostic. So you're wrong: operator ! can yield compile time diagnostics if the semantics are suitably chosen; Greg simply didn't chose the right semantics. -- John (Max) Skaller, mailto:skaller@maxtal.com.au 10/1 Toxteth Rd Glebe NSW 2037 Australia voice: 61-2-9660-0850 homepage: http://www.maxtal.com.au/~skaller download: ftp://ftp.cs.usyd.edu/au/jskaller From skaller@maxtal.com.au Mon Jan 24 13:31:37 2000 From: skaller@maxtal.com.au (skaller) Date: Tue, 25 Jan 2000 00:31:37 +1100 Subject: [Types-sig] how to do a typecase? (was: Static typing: Towards closure?) References: <000001bf663a$7ad551c0$022d153f@tim> Message-ID: <388C5439.5732A3E3@maxtal.com.au> Tim Peters wrote: > "The rules" are spelled out in the Haskell reference. I downloaded the > ocaml reference, and after failing to figure out *its* rules, bumped into > this paragraph: > > No attempt has been made at mathematical rigor: words are > employed with their intuitive meaning, without further > definition. As a consequence, the typing rules have been > left out, by lack of the mathematical framework required > to express them, while they are definitely part of a full > formal definition of the language. > > That's a real help . Every release has improved typing. There's a small but responsive mailing list, so that the rules are reasonably well explained by asking the developers questions. -- John (Max) Skaller, mailto:skaller@maxtal.com.au 10/1 Toxteth Rd Glebe NSW 2037 Australia voice: 61-2-9660-0850 homepage: http://www.maxtal.com.au/~skaller download: ftp://ftp.cs.usyd.edu/au/jskaller From skaller@maxtal.com.au Mon Jan 24 13:38:27 2000 From: skaller@maxtal.com.au (skaller) Date: Tue, 25 Jan 2000 00:38:27 +1100 Subject: [Types-sig] how to do a typecase? (was: Static typing: Towards closure?) References: <3889BE8B.12A0618F@maxtal.com.au> <000101bf651d$f21850c0$132d153f@tim> <14475.23900.397638.971158@bitdiddle.cnri.reston.va.us> <388B7FF7.5DBB6773@maxtal.com.au> <14475.53646.954417.163189@bitdiddle.cnri.reston.va.us> Message-ID: <388C55D3.28FD7C2C@maxtal.com.au> Jeremy Hylton wrote: > JS> previous ones] but it isn't clear how to detect this in general > JS> -- without eliminating useful (tractable) use of type recursion > JS> such as: > > JS> type node = Node of node | Int of int > > It isn't clear to me either, but I see that in Ocaml I can > successfully evaluate: > "type node = Node of node | Int of int" > but not > "let rec f () = f". > This suggests to me that it is possible to allow recursive types > without having a solution to latter problem. The thing here is to observe that none of us are type theorists. Well, amateur ones perhaps :-) So we should not byte off more than we can Choo. :-) -- John (Max) Skaller, mailto:skaller@maxtal.com.au 10/1 Toxteth Rd Glebe NSW 2037 Australia voice: 61-2-9660-0850 homepage: http://www.maxtal.com.au/~skaller download: ftp://ftp.cs.usyd.edu/au/jskaller From skaller@maxtal.com.au Mon Jan 24 13:47:56 2000 From: skaller@maxtal.com.au (skaller) Date: Tue, 25 Jan 2000 00:47:56 +1100 Subject: [Types-sig] how to do a typecase? (was: Static typing: Towards closure?) References: <000201bf661e$d57f3940$902d153f@tim> Message-ID: <388C580C.73889BAF@maxtal.com.au> Tim Peters wrote: > > [John Skaller, on def f(): return f] > > ... > > The question is: what will a type checker do, > > seeing this function? Will it loop forever trying to compute > > the type of f? > > I still won't call this a "type checking" problem; it's a (potential) > problem for a type *inferencer*, but: > > > The answer is: probably. And we must allow this if type > > declations are optional. The only way to disallow the above > > function is to require everything be explicitly typed, in > > which case there is no way to name the type of the function > > above. > > Bingo. Checking that use matches declaration is easier than inferring > declaration from use. "Mere checking" is hard enough for a first version, > though! I don't think these two things are separable. Consider: decl x : int y = 1 x = y How can you check y = x, if you do not infer the type of y? The RHS of an assignment is an EXPRESSION: the type of the expression MUST be deduced in order to check that it is compatible with the LHS variable. In the very simplest of cases, where the types of all identifiers (including all functions) are known and monomorphic, a single bottom up pass will compute the type of any expression. This is one sense in which Guido's 'checked module' idea makes sense, because it requires precisely that -- provided unchecked modules are not imported. As soon as ANY untyped identifier enters the picture, we're stuffed. It is NOT good enough to assume the type is 'ANY', since that is guarranteed to cause a type error in almost every possible use. [That is, almost all functions/expressions propogate type information: their type depends on the types of the component subexpressions] -- John (Max) Skaller, mailto:skaller@maxtal.com.au 10/1 Toxteth Rd Glebe NSW 2037 Australia voice: 61-2-9660-0850 homepage: http://www.maxtal.com.au/~skaller download: ftp://ftp.cs.usyd.edu/au/jskaller From paul@prescod.net Mon Jan 24 02:39:04 2000 From: paul@prescod.net (Paul Prescod) Date: Sun, 23 Jan 2000 20:39:04 -0600 Subject: [Types-sig] Interfaces should be on the table Message-ID: <388BBB48.10E04A9E@prescod.net> I think that interfaces should be on the table for version 1 of the type system for the following reason. The three ways that static type checking could most go "against the grain" of Python and thus destroy its beauty are: * they become mandatory (or contagious) so that beginners had to learn them for trivial operations * they stop people from doing really dynamic stuff in Python. (I'm happy to restrict really dynamic stuff in *checked code* but not in Python itself) * they destroy the awesome genericity that has been growing throughout the Python library wherein mappings can be used anywhere dictionaries can be, sequences anywhere lists can be and so forth. When I hear people get nervous about type safety it is almost always these three things that they are nervous about (and sometimes a vague syntactic sense that anything with type declarations "is not Python"). Practically speaking, there is no reason to take interfaces off the table. They are simpler than parameterized types (for example) and are implicit in any discussion of the distinction between subtyping and subclassing. Which brings me to... > After reading part of [Bruce96]: maybe we need to separate the notion > of subtyping and subclassing. This is precisely what interfaces do and they do it in a clear, unambiguous, syntactic way. It is through interfaces that Sather gets around Eiffel's covariance "issues" and Java gets around difficulties of multiple inheritance in deeply nested class libraries. -- Paul Prescod - ISOGEN Consulting Engineer speaking for himself Earth will soon support only survivor species -- dandelions, roaches, lizards, thistles, crows, rats. Not to mention 10 billion humans. - Planet of the Weeds, Harper's Magazine, October 1998 From paul@prescod.net Mon Jan 24 02:34:06 2000 From: paul@prescod.net (Paul Prescod) Date: Sun, 23 Jan 2000 20:34:06 -0600 Subject: [Types-sig] how to do a typecase? (was: Static typing: Towards closure?) References: <200001211633.LAA29205@eric.cnri.reston.va.us> <14472.38177.65905.81604@goon.cnri.reston.va.us> <3888C146.FCABD752@maxtal.com.au> <14472.54111.416650.255321@goon.cnri.reston.va.us> Message-ID: <388BBA1E.E7718D5A@prescod.net> Jeremy Hylton wrote: > > ... > > My argument restated briefly is that we should clearly specify the > type system for Python, so that we can verify that typecheckers > faithfully implement the type rules. I think that this should be non-negotiable. My proposals have been couched in standards-eze to allow exactly this level of interoperability. -- Paul Prescod - ISOGEN Consulting Engineer speaking for himself Earth will soon support only survivor species -- dandelions, roaches, lizards, thistles, crows, rats. Not to mention 10 billion humans. - Planet of the Weeds, Harper's Magazine, October 1998 From scott@chronis.pobox.com Mon Jan 24 16:08:20 2000 From: scott@chronis.pobox.com (scott) Date: Mon, 24 Jan 2000 11:08:20 -0500 Subject: [Types-sig] Type-checked code calling unchecked code In-Reply-To: <388C5287.D7C30D52@maxtal.com.au> References: <200001191722.MAA19286@eric.cnri.reston.va.us> <388B8576.618B4AAD@prescod.net> <200001240133.UAA01094@eric.cnri.reston.va.us> <20000124005706.A98461@chronis.pobox.com> <388C5287.D7C30D52@maxtal.com.au> Message-ID: <20000124110820.A1419@chronis.pobox.com> On Tue, Jan 25, 2000 at 12:24:23AM +1100, skaller wrote: > scott wrote: > > > > With Greg, you could write > > > > > > i = j() ! int > > > > > > > Just wanted to point out that in the case of '!', you get a runtime > > error if you are wrong. in the case of typecase, you get a compile > > time error if it's wrong (typewise, of course). I feel like I'm > > repeating myself re: the runtime errors associated with !, but I think > > it's important that it be pointed out, since the goal as I understand > > it is greater compile time safety, and since the issue hasn't really > > been addressed at all. > > Greg Stein's semantics are NOT the ones I advocate -- > but the syntax is fine. I define > > x ! y > > as an assertion, and like all assertions in Vyper -- and indeed in > Python > -- if they are wrong, then the language translator is NOT required > to raise a TypeError, in fact, it can do anything it wants. > In particular, it can ASSUME that the assertion holds, > and optimise accordingly, and it can PROVE (perhaps) that the > assertion is false, and issue a compile time diagnostic. > > So you're wrong: operator ! can yield compile time diagnostics > if the semantics are suitably chosen; Greg simply didn't chose the > right semantics. Ick. either way, you are using this syntax to push what a type checker should catch at compile time to runtime problems; regardless if it 'can' cause compile time diagnostics *will* it? The point of a type checker at this stage of development IMO is to have safer code at compile time. Either semantic for this operator doesn't seem to me to work toward that goal. The issue of this operator failing to work toward the goal of compile time safety is what needs to be addressed. Stating that a program is illegal since it raises an exception at runtime does not address this issue. Stating that the operator may be assumed true by a translator does not address this issue. If someone can show that it doesn't reduce compile time safety compared to the systems that don't have it, then it won't be an issue. As it stands, it is a problem, and needs to be addressed. scott From skaller@maxtal.com.au Mon Jan 24 21:09:13 2000 From: skaller@maxtal.com.au (skaller) Date: Tue, 25 Jan 2000 08:09:13 +1100 Subject: [Types-sig] Type-checked code calling unchecked code References: <200001191722.MAA19286@eric.cnri.reston.va.us> <388B8576.618B4AAD@prescod.net> <200001240133.UAA01094@eric.cnri.reston.va.us> <20000124005706.A98461@chronis.pobox.com> <388C5287.D7C30D52@maxtal.com.au> <20000124110820.A1419@chronis.pobox.com> Message-ID: <388CBF79.55F809DB@maxtal.com.au> scott wrote: > Ick. either way, you are using this syntax to push what a type > checker should catch at compile time to runtime problems; ?? > regardless > if it 'can' cause compile time diagnostics *will* it? The point of a > type checker at this stage of development IMO is to have safer code at > compile time. Either semantic for this operator doesn't seem to me to > work toward that goal. The issue of this operator failing to work > toward the goal of compile time safety is what needs to be addressed. I really have no idea what you're talking about. > Stating that a program is illegal since it raises an exception at > runtime does not address this issue. Stating that the operator may be > assumed true by a translator does not address this issue. If someone > can show that it doesn't reduce compile time safety compared to the > systems that don't have it, then it won't be an issue. As it stands, > it is a problem, and needs to be addressed. And now I have even less idea what you're talking about. To me, it is all very simple. Greg's operator ! is a major advance because it looks nice -- it's Pythonic. In Vyperi, the interpreter, a TypeError is raised. Every time there is one. In an as yet unwritten compiler Vyperc, the extra information will be used to detect type errors and issue a diagnostic, and to optimised the generated code. Therefore, it furthers the aims OPT and ERR. It is quite true that a Python program annotated with an occasional ! operator cannot be assumed to be type safe just because it has been run through the compiler, nor can one assume that maximally optimal code is being generated; but you CAN be sure that Vyperi will raise a TypeError at every annotated point of control, and you CAN try to make the code run faster, or provide better, earlier, diagnostics by adding additional uses of operator ! in various places. It is possible to generate a report showing which types have been infered by the type checking part of the compiler, and thus to know how to firm up your type specifications if desired. The simple facts are that this operator fits naturally into Python, is trivial to implement in an interpreter, and provides excellent opportunities to issue early diagnostics and optimise code. Those are the stated aims of this group. Some people are trying to do better, and obtain a system where type safety can be _guarranteed_; plain use of operator ! does not do this. Nor does ANY scheme in ANY (Turing complete) language provide complete safety. The notion of 'type safe' is relative: the amount of safety that it provides (when known to be true of a program) varies depending on the strength of the type system. For example, in ML, you can know something is a list of X, but if you are zipping two lists together, and they're not the same length, you get a run time error. But in C++, you will get a compile time error zipping two arrays of type Array because the length is made part of the type. This is a problem in ML, addressed in FISh by compile time shape analysis. So what I'm trying to say is that 'type-safe' is not a holy grail, especially in Python: the guarrantee is easy to obtain: EVERY python object has type PyObject, so ALL python programs are ALREADY guarranteed to be type safe. Do you follow? The type system 'everything is PyObject' is easy to check (def f(): return 1), and is enough to prevent core dumps. The declarative syntax is simple (very simple: there is none). But the type system is weak compared to other languages. We can make it stronger. But there is no fixed, magical, specification of typing: there is just language, and what can be deduced by a particular algorithm. The KEY is to (a) add syntax to specify typing, and (b) to permit the type information to be used to issue early diagnostics (ERR) and optimise code (OPT). Greg's operator ! adds nice syntax, and it allows both ERR and OPT. It fails to guarrantee type correctness at compile time. So what? As I explained, this doesn't mean anything anyhow: failing to exclude run time errors is always a property of static analysers --- they can help, they can even guarrantee _certain_ properties, but they can't eliminate all errors (at least with todays state of knowledge). -- John (Max) Skaller, mailto:skaller@maxtal.com.au 10/1 Toxteth Rd Glebe NSW 2037 Australia voice: 61-2-9660-0850 homepage: http://www.maxtal.com.au/~skaller download: ftp://ftp.cs.usyd.edu/au/jskaller From scott@chronis.pobox.com Tue Jan 25 00:49:25 2000 From: scott@chronis.pobox.com (scott) Date: Mon, 24 Jan 2000 19:49:25 -0500 Subject: [Types-sig] evaluating '!' [subject changed] In-Reply-To: <388CBF79.55F809DB@maxtal.com.au> References: <200001191722.MAA19286@eric.cnri.reston.va.us> <388B8576.618B4AAD@prescod.net> <200001240133.UAA01094@eric.cnri.reston.va.us> <20000124005706.A98461@chronis.pobox.com> <388C5287.D7C30D52@maxtal.com.au> <20000124110820.A1419@chronis.pobox.com> <388CBF79.55F809DB@maxtal.com.au> Message-ID: <20000124194925.A68501@chronis.pobox.com> On Tue, Jan 25, 2000 at 08:09:13AM +1100, skaller wrote: > scott wrote: > > > Ick. either way, you are using this syntax to push what a type > > checker should catch at compile time to runtime problems; [...] > > > regardless > > if it 'can' cause compile time diagnostics *will* it? The point of a > > type checker at this stage of development IMO is to have safer code at > > compile time. Either semantic for this operator doesn't seem to me to > > work toward that goal. The issue of this operator failing to work > > toward the goal of compile time safety is what needs to be addressed. > [...] > > Stating that a program is illegal since it raises an exception at > > runtime does not address this issue. Stating that the operator may be > > assumed true by a translator does not address this issue. If someone > > can show that it doesn't reduce compile time safety compared to the > > systems that don't have it, then it won't be an issue. As it stands, > > it is a problem, and needs to be addressed. [...] > > To me, it is all very simple. Greg's operator ! is a major > advance because it looks nice -- it's Pythonic. In Vyperi, > the interpreter, a TypeError is raised. Every time there is one. 'looks nice' is very much an opinion thing. IMO, it doesn't look nice, but that's not a very important aspect of the problem stated above. > > In an as yet unwritten compiler Vyperc, the extra information > will be used to detect type errors and issue a diagnostic, > and to optimised the generated code. > > Therefore, it furthers the aims OPT and ERR. OK. I see that, but it doesn't get us very far with a comparison of systems without a !-like construct and those with one. > > It is quite true that a Python program annotated with an > occasional ! operator cannot be assumed to be type safe just because > it has been run through the compiler, nor can one assume that maximally > optimal code is being generated; Yes, this is a problem with a system that uses a !-like construct. [.. skaller repeats the plausibilty of using ! to help w/ OPT and ERR ] > The simple facts are that this operator fits naturally into > Python, is trivial to implement in an interpreter, and provides > excellent opportunities to issue early diagnostics and optimise code. 1) I wouldn't classify 'fits naturally into Python' as a simple fact. Sounds very much like opinion to me. 2) I wouldn't classify the opportunities as excellent until they are compared with proposed systems without a !-like construct. Gotta be careful with what we call facts. > > Those are the stated aims of this group. And there have been many affirmations that OPT should come later, and be built on top of the most solid system ERR-wise that we can come up with. I think we all already know that with a more stringent type system we will be able to do a lot with optimization. > Some people are trying to do better, and obtain a system where > type safety can be _guarranteed_; plain use of operator ! does not > do this. I would restate this as 'some people are trying to do better and obtain a system where type safety is a more uniform and predictable and specifiable concept'. Maybe we can even come up with some things that can be guaranteed. > Nor does ANY scheme in ANY (Turing complete) language provide > complete safety. The notion of 'type safe' is relative: the amount > of safety that it provides (when known to be true of a program) > varies depending on the strength of the type system. [... example followed with ML & array lengths] OK, I'll trust you on that, as it seems true from my own work in this area. We haven't really gotten anywhere comparing type systems with a !-like construct and without, though. If we take this as true, all we have said is that all type systems cannot provide complete type safety. This point alone fails to explore any of the relative effects of using a !-like construct compared to not using one. I know of one such differentiating comparison: type systems in which !-like constructs are used will lack in the degree of safety that can be known at compile time by a type checking program in so far as those constructs are used in a way that the type checking program cannot prove as either correct or incorrect. In other words, any degree of compile time safety we can can accomplish will always exclude all usages of !-like constructs that are undecidable by the type-checker. Well, the more compile time safety, the better. We both agree on that. If we leave out a !-like construct, we will have opportunities to produce a greater degree of compile time safety, because a !-like construct introduces a specific limitation in compile time safety that does not exist in other systems, such as Guido's initial proposal. > > So what I'm trying to say is that 'type-safe' is not a > holy grail >, especially in Python: the guarrantee is easy to obtain: > EVERY python object has type PyObject, so ALL python programs > are ALREADY guarranteed to be type safe. > > Do you follow? yes. >The type system 'everything is PyObject' > is easy to check (def f(): return 1), and is enough to prevent > core dumps. The declarative syntax is simple (very simple: there is > none). > But the type system is weak compared to other languages. > > We can make it stronger. But there is no fixed, magical, > specification of typing: there is just language, and what can > be deduced by a particular algorithm. Yes, and if the language has a !-like construct about which the algo cannot decide, then we have made it weaker than we might. > > The KEY is to (a) add syntax to specify typing, > and (b) to permit the type information to be used to issue > early diagnostics (ERR) and optimise code (OPT). I'd opt to put the OPT off until we've got ERR. We'll be able to optimize plenty if we know enough to provide reasonable compile time error reporting. > > Greg's operator ! adds nice syntax I don't think so, but that's just another opinion on aesthetics. >, and it allows > both ERR and OPT. It fails to guarrantee type correctness > at compile time. So what? As I explained, this doesn't > mean anything anyhow: failing to exclude run time errors > is always a property of static analysers --- they can help, > they can even guarrantee _certain_ properties, but they can't > eliminate all errors (at least with todays state of knowledge). Other static type checking systems do better, though. I'd rather not limit the ability of the checker to attain compile time safety in this way, because it debilitates the type checker's potential to provide compile time error reports in a rather fundamental way (described above). scott From gstein@lyra.org Tue Jan 25 00:54:53 2000 From: gstein@lyra.org (Greg Stein) Date: Mon, 24 Jan 2000 16:54:53 -0800 (PST) Subject: [Types-sig] how to do a typecase? In-Reply-To: <388C580C.73889BAF@maxtal.com.au> Message-ID: On Tue, 25 Jan 2000, skaller wrote: > Tim Peters wrote: > > [John Skaller, on def f(): return f] > > > ... > > > The question is: what will a type checker do, > > > seeing this function? Will it loop forever trying to compute > > > the type of f? > > > > I still won't call this a "type checking" problem; it's a (potential) > > problem for a type *inferencer*, but: > > > > > The answer is: probably. And we must allow this if type > > > declations are optional. The only way to disallow the above > > > function is to require everything be explicitly typed, in > > > which case there is no way to name the type of the function > > > above. > > > > Bingo. Checking that use matches declaration is easier than inferring > > declaration from use. "Mere checking" is hard enough for a first version, > > though! > > I don't think these two things are separable. > Consider: > > decl x : int > y = 1 > x = y > > How can you check y = x, if you do not infer the type of y? > The RHS of an assignment is an EXPRESSION: the type of the > expression MUST be deduced in order to check that it is compatible > with the LHS variable. No big deal. This is where Tim's point about using pragmatism, hackery, or whatever, to get the problem solved. The "x = y" is all about tracking the types of expression values. Inferring a result type for f() is not the same as tracking a result. As you point out, the declaration is something like: def f() -> Any def f() -> def() -> Any def f() -> def() -> def() -> Any ... Given a declaration, we can show that the code checks out properly. In the "x = y" case, we simply deduce the expression types. For the function, we just state "we don't deduce/infer return types: you must declare them." Problem solved. This still begs the question, "are there other, similar examples?" >... > As soon as ANY untyped identifier enters the picture, we're stuffed. > It is NOT good enough to assume the type is 'ANY', since that is > guarranteed to cause a type error in almost every possible use. > [That is, almost all functions/expressions propogate type information: > their type depends on the types of the component subexpressions] Not necessarily true. The type-assert operator serves to "narrow" the available types down to a specific type. decl x: int y = f() # y has type ANY because we don't have a decl for f() x = y!int # we narrow y down to an int Of course, this does imply that a runtime assertion *may* occur, but we're assuming the programmer knows something the checker doesn't and is using the type-assert to tell the checker about it. The assertion is also double-checking the programmer's belief. Cheers, -g -- Greg Stein, http://www.lyra.org/ Return-Path: Delivered-To: types-sig@dinsdale.python.org Received: from python.org (parrot.python.org [132.151.1.90]) by dinsdale.python.org (Postfix) with ESMTP id 014E11CE8F for ; Wed, 19 Jan 2000 14:48:52 -0500 (EST) Received: from cnri.reston.va.us (ns.CNRI.Reston.VA.US [132.151.1.1] (may be forged)) by python.org (8.9.1a/8.9.1) with ESMTP id OAA10627 for ; Wed, 19 Jan 2000 14:48:51 -0500 (EST) Received: from kaluha.cnri.reston.va.us (kaluha.cnri.reston.va.us [132.151.7.31]) by cnri.reston.va.us (8.9.1a/8.9.1) with ESMTP id OAA24801 for ; Wed, 19 Jan 2000 14:48:51 -0500 (EST) Received: from eric.cnri.reston.va.us (eric.cnri.reston.va.us [10.27.10.23]) by kaluha.cnri.reston.va.us (8.9.1b+Sun/8.9.1) with ESMTP id OAA21898 for ; Wed, 19 Jan 2000 14:50:02 -0500 (EST) Received: from CNRI.Reston.VA.US (localhost [127.0.0.1]) by eric.cnri.reston.va.us (8.9.3+Sun/8.9.1) with ESMTP id OAA19639 for ; Wed, 19 Jan 2000 14:50:01 -0500 (EST) Resent-Message-Id: <200001191950.OAA19639@eric.cnri.reston.va.us> Replied: Wed, 19 Jan 2000 14:49:02 -0500 Replied: ""types-sig@python.org " Received: from cnri.reston.va.us (ns.CNRI.Reston.VA.US [132.151.1.1]) by kaluha.cnri.reston.va.us (8.9.1b+Sun/8.9.1) with SMTP id OAA21707 for ; Wed, 19 Jan 2000 14:25:49 -0500 (EST) Received: from ns1.cnri.reston.va.us (host202 [132.151.1.202]) by cnri.reston.va.us (8.9.1a/8.9.1) with ESMTP id OAA24493; Wed, 19 Jan 2000 14:24:32 -0500 (EST) Received: from goon.cnri.reston.va.us.cnri.reston.va.us (goonx [10.27.30.39]) by ns1.cnri.reston.va.us (8.8.8+Sun/8.8.8) with SMTP id OAA17821; Wed, 19 Jan 2000 14:25:42 -0500 (EST) MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Message-ID: <14470.4022.551614.971388@goon.cnri.reston.va.us> Date: Wed, 19 Jan 2000 14:25:42 -0500 (EST) From: Jeremy Hylton To: Guido van Rossum Cc: "types-sig@python.org Greg Stein" , Paul Prescod In-Reply-To: <200001191902.OAA19487@eric.cnri.reston.va.us> References: <200001191729.MAA19324@eric.cnri.reston.va.us> <14470.2448.957015.160023@goon.cnri.reston.va.us> <200001191902.OAA19487@eric.cnri.reston.va.us> X-Mailer: VM 6.75 under 21.1 (patch 7) "Biscayne" XEmacs Lucid Reply-To: jeremy@CNRI.Reston.VA.US Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii Resent-To: types-sig@python.org Resent-Date: Wed, 19 Jan 2000 14:50:01 -0500 Resent-From: Guido van Rossum Content-Transfer-Encoding: 7bit Subject: [Types-sig] Re: puzzle about subtyping and parameterization Sender: types-sig-admin@python.org Errors-To: types-sig-admin@python.org X-BeenThere: types-sig@python.org X-Mailman-Version: 1.2 (experimental) Precedence: bulk List-Id: Special Interest Group on the Python type system >>>>> "GvR" == Guido van Rossum writes: [in an earlier msg guido proposed some subtype rules] GvR> int <: int|long <: integer <: number <: any The subtype rules define when it is legal to assign a particular value to a variable or pass it as a parameter to a function. In general, an assignment of a value of type S to a variable of T is legal if S is a subtype of T, written S <: T. [I replied] >> This is just a nit, but it seems mighty confusing to have two >> different types with what is effectively the same name -- int and >> integer. For most people, int is an abbreviation that means >> integer. GvR> Good point. I've done the same with 'str' (8-bit string) and GvR> 'string' (8-bit or Unicode string). Same complaint. No idea what the right answer is. GvR> But what else to name it? We need a name for the abstract GvR> interface that int, long and custom integers share. (And the GvR> semantics that division truncates. :-( ) Perhaps 'integral'? I thought I had a solution until you mentioned strings :-(. For numeric types, I was going to propose that number be used most of the time, and that 1/2 == .5 and that 2**45 doesn't raise an OverflowError. If most programs use this version of number, then we can afford to give funny names to what you have called 'int' and 'integer' because most programs won't use them. How about 'fixnum' for what is now IntType and 'int' for IntType and LongType? GvR> I'm reluctant to introduce case differences; many people don't GvR> notice the difference between Int and int anyway. Yikes! I'm worried that you're only "reluctant" and not "morally opposed." Jeremy From tim_one@email.msn.com Tue Jan 25 07:34:26 2000 From: tim_one@email.msn.com (Tim Peters) Date: Tue, 25 Jan 2000 02:34:26 -0500 Subject: [Types-sig] how to do a typecase? (was: Static typing: Towards closure?) In-Reply-To: <388C580C.73889BAF@maxtal.com.au> Message-ID: <000401bf6706$9b6c6460$632d153f@tim> [Tim] >> Bingo. Checking that use matches declaration is easier than >> inferring declaration from use. "Mere checking" is hard enough >> for a first version, though! [John Skaller] > I don't think these two things are separable. > Consider: > > decl x : int > y = 1 > x = y > > How can you check y = x, if you do not infer the type of y? You're still in the inference game here. Add decl y: int too and checking "x = y" is a trivial local operation. It does take *some* smarts to verify that "y = 1" is cool. > The RHS of an assignment is an EXPRESSION: the type of the > expression MUST be deduced in order to check that it is > compatible with the LHS variable. Sure. The difference is that in the presence of explicit declarations, control flow is taken out of the picture (there is no (relevant) control flow *within* a Python expression). Only part of the undecidability of rich type systems is due to the undecidability of hairy unifications of type expressions; another part is due to the static undecibability of control flow; I suspect (but don't know) that part of the reason Haskell's inference is bulletproof despite the richness of its type system is that, as a functional language, it has no control flow to worry about (there are no *dataflow* equations to solve, only type unifications; inference in Python suffers full-blown exposure to both). There is certainly no hope for building an inference engine if there's no hope for solving the static "mere checking" sub-problem. Solving *only* the latter is *sufficient* for ERR and OPT (and is the whole point of DOC), although it's not as *convenient* as you (Guido either, for that matter) would like. Nevertheless, it still needs to be solved, as it's the foundation on which everything else stands (or falls). Every time inter-statement inference gets dragged into this, it just strikes me as distraction from getting the initial design specified and the initial work done. > In the very simplest of cases, where the types of all identifiers > (including all functions) are known Which is indeed the only thing I think we should be burning time on now. > and monomorphic, Scott makes a decent case for restricting attention to that too at first. > a single bottom up pass will compute the type of any expression. So let's do that, get the code the out there, and generalize later. > ... > As soon as ANY untyped identifier enters the picture, we're > stuffed. The extent to which that's true also defines the extent to which it's wise to forbid the possibility in the first cut. Most Python programmers likely come from C, C++ and Java, where "declare everything" is absolutely required. They won't feel that a half-assed partial inference scheme is "a present", they'll just go "huh?" <0.5 wink>. trying-to-do-better-is-a-mistake-so-long-as-we-can't-even-do- half-as-well-ly y'rs - tim From Vladimir.Marangozov@inrialpes.fr Tue Jan 25 10:40:24 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Tue, 25 Jan 2000 11:40:24 +0100 (CET) Subject: [Types-sig] how to do a typecase? (was: Static typing: Towards closure?) In-Reply-To: <000401bf6706$9b6c6460$632d153f@tim> from "Tim Peters" at Jan 25, 2000 02:34:26 AM Message-ID: <200001251040.LAA04242@python.inrialpes.fr> Tim Peters wrote: > > There is certainly no hope for building an inference engine if there's no > hope for solving the static "mere checking" sub-problem. Solving *only* the > latter is *sufficient* for ERR and OPT (and is the whole point of DOC), > although it's not as *convenient* as you (Guido either, for that matter) > would like. > > Nevertheless, it still needs to be solved, as it's the foundation on which > everything else stands (or falls). Every time inter-statement inference > gets dragged into this, it just strikes me as distraction from getting the > initial design specified and the initial work done. Ah. Let me plug that it's nice to see the discussion changing its tone from "type constraining" to "type checking", although the two aspects are tightly related to each other. With the latest proposals, I think there's enough material already for experimenting with a basic type checker which could perform some ERR. nothing-constructive-ly y'rs -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From skaller@maxtal.com.au Tue Jan 25 13:00:07 2000 From: skaller@maxtal.com.au (skaller) Date: Wed, 26 Jan 2000 00:00:07 +1100 Subject: [Types-sig] Re: evaluating '!' [subject changed] References: <200001191722.MAA19286@eric.cnri.reston.va.us> <388B8576.618B4AAD@prescod.net> <200001240133.UAA01094@eric.cnri.reston.va.us> <20000124005706.A98461@chronis.pobox.com> <388C5287.D7C30D52@maxtal.com.au> <20000124110820.A1419@chronis.pobox.com> <388CBF79.55F809DB@maxtal.com.au> <20000124194925.A68501@chronis.pobox.com> Message-ID: <388D9E57.BCEE9CFC@maxtal.com.au> scott wrote: > > The simple facts are that this operator fits naturally into > > Python, is trivial to implement in an interpreter, and provides > > excellent opportunities to issue early diagnostics and optimise code. > > 1) I wouldn't classify 'fits naturally into Python' as a simple fact. > Sounds very much like opinion to me. My opinion is based on actually implementing it. It didn't take long. It is easy to use. It fits into the grammar with trivial changes, introduces no ambiguities, and the interpretive semantics were implemented in a few lines of code. > 2) I wouldn't classify the opportunities as excellent until they are > compared with proposed systems without a !-like construct. > > Gotta be careful with what we call facts. Fair enough: I haven't implemented any of the other proposals. I don't know how. They don't seem to be 'simple'. > > Some people are trying to do better, and obtain a system where > > type safety can be _guarranteed_; plain use of operator ! does not > > do this. > > I would restate this as 'some people are trying to do better and obtain > a system where type safety is a more uniform and predictable and > specifiable concept'. Maybe we can even come up with some things that > can be guaranteed. Yes, that's a better statement. > > Nor does ANY scheme in ANY (Turing complete) language provide > > complete safety. The notion of 'type safe' is relative: the amount > > of safety that it provides (when known to be true of a program) > > varies depending on the strength of the type system. > > [... example followed with ML & array lengths] > > OK, I'll trust you on that, as it seems true from my own work in this > area. We haven't really gotten anywhere comparing type systems with a > !-like construct and without, though. My argument in favour of ! is that it can be implemented now, easily, and provides considerable benefits. This does not mean it is the best that can be done: clearly it is not a complete system. But there is a danger of getting lost trying to build more complete systems for a language like Python, without a coherent theoretical base. ML-style languages have enough trouble, and they have world class mathematicians developing the theory. Python isn't the same kind of system, there isn't much theory I've seen on integrating static and dynamic typing. Hence .. it may make sense to be conservative. But I agree we should investigate multiple systems. > This point alone fails to explore any of the relative effects of using > a !-like construct compared to not using one. I know of one such > differentiating comparison: > > type systems in which !-like constructs are used will lack in the > degree of safety that can be known at compile time by a type > checking program in so far as those constructs are used in a way > that the type checking program cannot prove as either correct or > incorrect. This is a true fact about !. But it is also true about the other proposals as well, to some degree. This is my point. The more 'stringent' systems are threatening in two ways: to be hard to implement (and use), and to threaten dynamic fluidity. This is why the argument in favour of interfaces, instead of types, is appealing -- it tends to preserve the genericity dynamics offers more than typing (which tends to be more specific). > Well, the more compile time safety, the better. We both agree on > that. I believe we agree on it, although I wouldn't have said it quite that way :-) >If we leave out a !-like construct, we will have opportunities > to produce a greater degree of compile time safety, I don't think so .. it is possible to implement operator ! AND any other static typing proposal, since one could just define operator ! as def exclamation(object, typ): if type(object) is typ: return object else : raise TypeError, "Excl assertion" Since the operator can be defined 'in' python, it is just syntactic sugar and cannot interfere with other proposals. [Technically, this is not quite right, since MY definition is not equivalent to the above code .. but my interpretive implementation is] > because a !-like > construct introduces a specific limitation in compile time safety that > does not exist in other systems, such as Guido's initial proposal. i do not see this. It introduces no limitations at all. It simply adds an extra kind of assertion. > Yes, and if the language has a !-like construct about which the algo > cannot decide, then we have made it weaker than we might. No: my existing implementation cannot decide at all, for ANY use, it always generates run time checks (at present). I do not see how this weakens anything, I could have written the checks in Python. > > The KEY is to (a) add syntax to specify typing, > > and (b) to permit the type information to be used to issue > > early diagnostics (ERR) and optimise code (OPT). > > I'd opt to put the OPT off until we've got ERR. Well, I think the point is to design something which enables both -- they're logically inseparable. > We'll be able to > optimize plenty if we know enough to provide reasonable compile time > error reporting. Agreed. > > Greg's operator ! adds nice syntax > > I don't think so, but that's just another opinion on aesthetics. There is style involved, but it is not _entirely_ a stylistic issue. Adding a binary operator fits in reasonably well (to almost any language!), compared to adding new compile time declarations .. that is a much bigger step. I'm NOT against it at all. But I'm concerned that we don't really know what we're doing .. this stuff is HARD. > Other static type checking systems do better, though. I'd rather not > limit the ability of the checker to attain compile time safety in this > way, But I do not see that there is any limitation here. Any more than 'assert'. -- John (Max) Skaller, mailto:skaller@maxtal.com.au 10/1 Toxteth Rd Glebe NSW 2037 Australia voice: 61-2-9660-0850 homepage: http://www.maxtal.com.au/~skaller download: ftp://ftp.cs.usyd.edu/au/jskaller From skaller@maxtal.com.au Tue Jan 25 13:15:35 2000 From: skaller@maxtal.com.au (skaller) Date: Wed, 26 Jan 2000 00:15:35 +1100 Subject: [Types-sig] how to do a typecase? (was: Static typing: Towards closure?) References: <000401bf6706$9b6c6460$632d153f@tim> Message-ID: <388DA1F7.36C8E04@maxtal.com.au> Tim Peters wrote: > > The RHS of an assignment is an EXPRESSION: the type of the > > expression MUST be deduced in order to check that it is > > compatible with the LHS variable. > > Sure. The difference is that in the presence of explicit declarations, > control flow is taken out of the picture (there is no (relevant) control > flow *within* a Python expression). Yes, there is: with 'and' and 'or', these are shortcut operators. Furthermore, function calls involve control flow :-) > Only part of the undecidability of rich > type systems is due to the undecidability of hairy unifications of type > expressions; another part is due to the static undecibability of control > flow; I suspect (but don't know) that part of the reason Haskell's inference > is bulletproof despite the richness of its type system is that, as a > functional language, it has no control flow to worry about (there are no > *dataflow* equations to solve, only type unifications; inference in Python > suffers full-blown exposure to both). Yes. That is my worry: I do not think it is possible to escape from inference. -- John (Max) Skaller, mailto:skaller@maxtal.com.au 10/1 Toxteth Rd Glebe NSW 2037 Australia voice: 61-2-9660-0850 homepage: http://www.maxtal.com.au/~skaller download: ftp://ftp.cs.usyd.edu/au/jskaller From skaller@maxtal.com.au Tue Jan 25 15:25:14 2000 From: skaller@maxtal.com.au (skaller) Date: Wed, 26 Jan 2000 02:25:14 +1100 Subject: [Types-sig] Typing References: <000401bf6706$9b6c6460$632d153f@tim> <388DA1F7.36C8E04@maxtal.com.au> Message-ID: <388DC05A.28DA4ED0@maxtal.com.au> I'd like to analyse what is going on. Please admit some simplifications for the sake of a clearer argument. 1. In a completely declaration free language (like Python 1.5), there are two ways to learn about typing: a) from literals (which have specified types) b) from builtin functions and operators (which have a set of possible types) Given only this information, it is possible to discover type errors by type inference. For example: x = y + 1 # x,y must be numbers, can't be strings z = x + "" # must be an error somewhere however there are cases where less is learned: a = b * c # b cannot be a file or dictionary, # could be tuple list or number z = b['x'] # error somewhere 2. Adding declarations has two benefits: it improves the opportunities for detecting errors, and it improves the quality of diagnostics (by assuming that the declaration is correct, we report a use as faulty) 3. If we add 'enough' declarations, we can detect all type errors at compile time easily. In particular, if every identifier is monomorphic (has exactly one non-variant type), we can infer the types of all expressions with a trivial bottom up calculation (easily implemented with a recursive descent). Please note that although the inference engine in this case is simple, we are still infering types. Unfortunately there are a number of reasons this simple scheme cannot work in Python. In the first place, variables can have different types at different times: I'll say these variables are polymorphic. Note that the variables have a definite type at a definite time, and we might be able to deduce the type. On the other hand, a variable can be monomorphic in the more general scheme (1) or (2) but we don't KNOW what the type is, only a set of possible types: this doesn't make the variable polymorphic (just our knowledge is incomplete). Now, let's compare, as scott asked, operator ! with variable declarations. Operator ! is less strict, it provides partial information at a single place, and refers to an expression, whereas the declaration is global and applies to an identifier. This means that the declarative form is much more definite .. and also more restrictive. Now, if we go to the extreme of Guido's idea, and require ALL identifiers in a 'checked' module be declared, then: IF the typing is monomorphic, a LOT of code will be unsuitable for checking, because a variable takes on more than one type: for example: for i in [1,'a']: print i On the other hand, if we permit variant types: decl i: int | string then we have a problem as follows: consider a function: decl f: (int -> (string | int)) | (string -> (float | long)) then the type of f(i) can be string, int, float or long. Type sets grow rapidly, and shrink less often -- particularly if we do not use more than the trivial bottom up inference scheme. If we want to catch _real_ errors, not fake ones, we must narrow the typing down with more powerful inferencing -- rather than issue wrong error messages. For example: for i in [0,1]: print [10,'ten'][i] + [20,'twenty'][i] In THIS code, a simple checker will find no error, but in THIS code: decl y : int | string decl x : int | string for i in [0,1]: x = [10,'ten'][i] y = [20,'twenty][i] print x + y we must issue an error, because the possibility that x is an int, and y is a string, and x + y would be wrong, exists .. and cannot be excluded by bottom up analysis. It is not definite there is an error .. and it is not definite that there is not. Now, the point is that in THIS case, we know there is a workaround, which perverts the readability of the code .. and readability is a source of reliability. I'm not sure that this will always be the case (since Python is not a functional language). I hope it is clear from this, that merely declaring identifiers doesn't remove the need for non-trivial inference. Checked modules will be useless if they cannot handle a wide variety of Python uses. People use python precisely _because_ such things are possible without having static type checking get in the way. I believe we will have to fall back on a looser system, in which optional type assertions are permitted, and it is permitted to reject necessarily wrong programs and/or assume that the programmer is not lying. I don't see, at present, how it is possible to introduce strong enough rules, that ALL type errors are eliminated, unless these rules remove so much of the dynamism that substitues for genericity, that a lot of programs will be 'broken'. To put this another way -- Gregs operator ! has the advantage that it is entirely optional, can be applied to variables or expressions, and can be implemented by issuing compile time errors, run time errors, or by optimisations which cause wrong programs to crash. The user has control of all this, and in particular, can attempt to debug a program --including existing untyped ones -- by adding more assertions, can attempt to optimise a program, by adding more assertions, and can do this for ANY program, module, or function. Because the application is localised and applies to arbitrary expressions, it can provide a rich source of information, without the clutter that mandatory variable typing can provide, and can, in fact, provide MORE information. Just consider: decl f: Any -> string | int f(x) ! string Here, knowing the type of x does NOT help determining the type of f(x). But Steins operator does. Note that if we write: decl z : string z = f(x) we must issue a type error diagnostic (as I understand it), because it isn't definite that f(x) will be a string (it could be an int). If I wanted to use a strictly typed language, I'd use one. Such as ocaml. I use python when that kind of typing gets in the way. I'm in favour of adding optional static typing constructions. I'm NOT in favour of trying to copy the style of existing statically typed languages -- why use python instead of them? This is one of the reasons I believe in type _inference_: it is possible to infer typing in a lot of cases, which means the programmer only needs to add _some_ type declarations to allow errors to be diagnosed. Perhaps even more important -- and I use ocaml this way -- it is possible to make a program more readable by adding SOME type declarations -- optional typing will be a great benefit compared with none, and possibly better than mandatory typing. Please note in closing that I make these claims without real substance -- it is mainly intuition that suggests to me that a conservative approach here is best. Type inference is hard -- but unlike statically type languages, in python it doesn't _have_ to be perfect. A poor inferencer can be compensated for by adding more assertions. This will not catch all errors at compile time. Indeed, an early implementation would catch them at run time (i.e., do no inferencing at all). -- John (Max) Skaller, mailto:skaller@maxtal.com.au 10/1 Toxteth Rd Glebe NSW 2037 Australia voice: 61-2-9660-0850 homepage: http://www.maxtal.com.au/~skaller download: ftp://ftp.cs.usyd.edu/au/jskaller From gstein@lyra.org Tue Jan 25 23:08:46 2000 From: gstein@lyra.org (Greg Stein) Date: Tue, 25 Jan 2000 15:08:46 -0800 (PST) Subject: [Types-sig] how to do a typecase? In-Reply-To: <000401bf6706$9b6c6460$632d153f@tim> Message-ID: What are you guys mumbling and bumbling about? My *prototype* already handles the code below. It knows "y" becomes and int, and it knows that "x" is being assigned an integer. In other words, the whole notion of "we should only be burning time on [declaring everything first]" is a moot point. We already have an example that performs "inter-statement inference", and we already have "code out there". We don't have to declare everything. Tim: you're trying to simplify where it isn't needed John: you're trying to make a non-existent problem harder Cheers, -g On Tue, 25 Jan 2000, Tim Peters wrote: > [Tim] > >> Bingo. Checking that use matches declaration is easier than > >> inferring declaration from use. "Mere checking" is hard enough > >> for a first version, though! > > [John Skaller] > > I don't think these two things are separable. > > Consider: > > > > decl x : int > > y = 1 > > x = y > > > > How can you check y = x, if you do not infer the type of y? > > You're still in the inference game here. Add > > decl y: int > > too and checking "x = y" is a trivial local operation. It does take *some* > smarts to verify that "y = 1" is cool. > > > The RHS of an assignment is an EXPRESSION: the type of the > > expression MUST be deduced in order to check that it is > > compatible with the LHS variable. > > Sure. The difference is that in the presence of explicit declarations, > control flow is taken out of the picture (there is no (relevant) control > flow *within* a Python expression). Only part of the undecidability of rich > type systems is due to the undecidability of hairy unifications of type > expressions; another part is due to the static undecibability of control > flow; I suspect (but don't know) that part of the reason Haskell's inference > is bulletproof despite the richness of its type system is that, as a > functional language, it has no control flow to worry about (there are no > *dataflow* equations to solve, only type unifications; inference in Python > suffers full-blown exposure to both). > > There is certainly no hope for building an inference engine if there's no > hope for solving the static "mere checking" sub-problem. Solving *only* the > latter is *sufficient* for ERR and OPT (and is the whole point of DOC), > although it's not as *convenient* as you (Guido either, for that matter) > would like. > > Nevertheless, it still needs to be solved, as it's the foundation on which > everything else stands (or falls). Every time inter-statement inference > gets dragged into this, it just strikes me as distraction from getting the > initial design specified and the initial work done. > > > In the very simplest of cases, where the types of all identifiers > > (including all functions) are known > > Which is indeed the only thing I think we should be burning time on now. > > > and monomorphic, > > Scott makes a decent case for restricting attention to that too at first. > > > a single bottom up pass will compute the type of any expression. > > So let's do that, get the code the out there, and generalize later. > > > ... > > As soon as ANY untyped identifier enters the picture, we're > > stuffed. > > The extent to which that's true also defines the extent to which it's wise > to forbid the possibility in the first cut. Most Python programmers likely > come from C, C++ and Java, where "declare everything" is absolutely > required. They won't feel that a half-assed partial inference scheme is "a > present", they'll just go "huh?" <0.5 wink>. > > trying-to-do-better-is-a-mistake-so-long-as-we-can't-even-do- > half-as-well-ly y'rs - tim > > > > _______________________________________________ > Types-SIG mailing list > Types-SIG@python.org > http://www.python.org/mailman/listinfo/types-sig > -- Greg Stein, http://www.lyra.org/ From tim_one@email.msn.com Wed Jan 26 00:01:29 2000 From: tim_one@email.msn.com (Tim Peters) Date: Tue, 25 Jan 2000 19:01:29 -0500 Subject: [Types-sig] how to do a typecase? (was: Static typing: Towards closure?) In-Reply-To: <388DA1F7.36C8E04@maxtal.com.au> Message-ID: <000c01bf6790$7f65e900$862d153f@tim> [John Skaller] >>> The RHS of an assignment is an EXPRESSION: the type of the >>> expression MUST be deduced in order to check that it is >>> compatible with the LHS variable. [Tim] >> Sure. The difference is that in the presence of explicit >> declarations, control flow is taken out of the picture (there >> is no (relevant) control flow *within* a Python expression). [John] > Yes, there is: with 'and' and 'or', these are shortcut > operators. Ah, you're in *that* mood today <0.7 wink>; OK, I'll try to expand on what the overlooked "(relevant)" might mean. Given type(X) and type(Y), the type of Python X and Y is type(X) | type(Y) The same is true of Python X or Y So to a mere checker, they're the same, and lead to nothing worse than a union type. Even better, in practice one almost never sees type(X) != type(Y) when these control structures are used, and a mere checker will usually be told that up front (or be a trivial deduction away from figuring it out). A bottom-up checker simply has no difficulties here; an inferencer has a much harder life (trying to dope out the types of X and Y individually given the combined constraint that it knows type(X)|type(Y) -- maybe, on a good day, when it's lucky). What "(relevant)" may mean is "does not create or contribute to a loop". That is, there are no loops within a Python expression ("and" and "or" or not), which appears to be essentially *why* a single bottom-up pass suffices for a mere checker. Stick a loop in the program (a recursion in the type equations), and you've suddenly got a fixed-point problem. Just checking whether *a* claimed set of types is a solution seems a whale of a lot easier than inferring *the* least fixed point to me. > Furthermore, function calls involve control flow :-) To a mere checker they don't -- the signatures are handed to it up front, and they may as well be replaced by constant literals of the same types. That they're "really functions" is simply irrelevant when you're not trying to do inference. All this certainly doesn't mean that "mere checking" is easy. Just *easier*. For example, it takes non-trivial flow analysis to have any hope of knowing that this "mere checking" function is declared correctly: def make42list(count: int, asstring: boolean=1) -> \ [int] | [str]: decl answer: [int] | [str] answer = [] decl i: int for i in range(count): answer.append(asstring and "42" or 42) return answer Everyone's (natural!) eagerness to get rid of the "decl i" is a distraction from addressing the *fundamental* problems we face. leave-the-trivial-crap-for-later-...-and-the-hard- crap-too-ly y'rs - tim From gstein@lyra.org Wed Jan 26 00:24:36 2000 From: gstein@lyra.org (Greg Stein) Date: Tue, 25 Jan 2000 16:24:36 -0800 (PST) Subject: [Types-sig] type checking (was: how to do a typecase?) In-Reply-To: <000c01bf6790$7f65e900$862d153f@tim> Message-ID: [ doesn't *anybody* ever want to update the subject line besides me? it does help with context here, knowing when a new thread has actually started... ] On Tue, 25 Jan 2000, Tim Peters wrote: >... Tim saying "no control flow in expressions", John responding: > [John] > > Yes, there is: with 'and' and 'or', these are shortcut > > operators. > > Ah, you're in *that* mood today <0.7 wink>; OK, I'll try to expand on what > the overlooked "(relevant)" might mean. Given type(X) and type(Y), the type > of Python > > X and Y > > is > > type(X) | type(Y) > > The same is true of Python > > X or Y > > So to a mere checker, they're the same, and lead to nothing worse than a > union type. Please see check.py in my prototype. Look for the _test() and _and_test() methods. You will see it already constructs a union for these expressions. > Even better, in practice one almost never sees type(X) != > type(Y) when these control structures are used, and a mere checker will > usually be told that up front (or be a trivial deduction away from figuring > it out). typedecl.union_typedecls() "compresses" an input list of type declarations into a result typedecl. It uses a supertype when possible, otherwise a union. For example: "any | int" becomes "any". > A bottom-up checker simply has no difficulties here; an inferencer > has a much harder life (trying to dope out the types of X and Y individually > given the combined constraint that it knows type(X)|type(Y) -- maybe, on a > good day, when it's lucky). Note that *any* type-checker must be able to compute the type of the RHS. What it does with the result is up to the checker. It can say the LHS becomes that type (no decl present), or it can check that the LHS accepts that type (decl present). I explained this key point about a month ago as a differentiating factor between Paul's ideas and my own... Specifically, that my approach was no more difficult than his, but that we took differing positions on policy. >... > > Furthermore, function calls involve control flow :-) > > To a mere checker they don't -- the signatures are handed to it up front, > and they may as well be replaced by constant literals of the same types. > That they're "really functions" is simply irrelevant when you're not trying > to do inference. Agreed. _check_function_call() in check.py doesn't check the arguments, but it *does* use the return type information. It also understands (class_type)() to be a constructor and returns an "instance" typedecl. > All this certainly doesn't mean that "mere checking" is easy. Just > *easier*. For example, it takes non-trivial flow analysis to have any hope > of knowing that this "mere checking" function is declared correctly: > > def make42list(count: int, asstring: boolean=1) -> \ > [int] | [str]: > decl answer: [int] | [str] > answer = [] > decl i: int > for i in range(count): > answer.append(asstring and "42" or 42) > return answer I agree that this is difficult. I know that my prototype would fail for two reasons: 1) it could not deduce that "asstring" is an invariant, and thus recognize that the and/or idiom would produce one type or the other in each iteration. 2) understand the notion of parameterized types and how .append() fits into that. I plan to make (2) work, but (1) will be a pain in the ass. I can certainly think of a few ways to go about it, but hoo boy! Later :-) > Everyone's (natural!) eagerness to get rid of the "decl i" is a distraction > from addressing the *fundamental* problems we face. Eager? Nah. I've already torched it. I'm working on filling in the rest of the stuff now. In fact, I've got a checkin to make and a release to do this evening. Cheers, -g -- Greg Stein, http://www.lyra.org/ From tim_one@email.msn.com Wed Jan 26 00:49:39 2000 From: tim_one@email.msn.com (Tim Peters) Date: Tue, 25 Jan 2000 19:49:39 -0500 Subject: [Types-sig] how to do a typecase? In-Reply-To: Message-ID: <001101bf6797$39e206a0$862d153f@tim> [Greg Stein] > What are you guys mumbling and bumbling about? The differences (if any) between inference and "mere checking". > My *prototype* already handles the code below. It knows > "y" becomes and int, and it knows that "x" is being > assigned an integer. Since John picked it to be the most trivial example of inter-stmt inference imaginable, I should hope so . > In other words, the whole notion of "we should only be > burning time on [declaring everything first]" is a moot > point. We already have an example that performs "inter-statement > inference", and we already have "code out there". We don't have > to declare everything. Understood, except we have a half-assed, ad hoc inferencer (not a comment on the quality of the work, but on its limitations) that's likely (*because* ad hoc) to confuse as much as it helps (precisely characterize the set of inferences it can make, in a way programmers feel confident they understand it fully -- be sure to address Jeremy's requirement that all Python inferencers have exactly the same capabilities). > Tim: you're trying to simplify where it isn't needed Actually, to me, the little time it's taken for us to exchange this little msg on the topic is just another bit taken away from fundamental problems. > John: you're trying to make a non-existent problem harder better-than-pretending-hard-problems-are-non-existent-ly y'rs - tim From skaller@maxtal.com.au Wed Jan 26 02:05:50 2000 From: skaller@maxtal.com.au (skaller) Date: Wed, 26 Jan 2000 13:05:50 +1100 Subject: [Types-sig] how to do a typecase? References: Message-ID: <388E567E.F688BA77@maxtal.com.au> Greg Stein wrote: > > What are you guys mumbling and bumbling about? My *prototype* already > handles the code below. It knows "y" becomes and int, and it knows that > "x" is being assigned an integer. > > In other words, the whole notion of "we should only be burning time on > [declaring everything first]" is a moot point. We already have an example > that performs "inter-statement inference", and we already have "code out > there". We don't have to declare everything. > > Tim: you're trying to simplify where it isn't needed > John: you're trying to make a non-existent problem harder I don't know about that: my problem is to convince Guido what the right thing to do is so I can upgrade Vyper. For me, that is a hard problem, because I don't know the answer. -- John (Max) Skaller, mailto:skaller@maxtal.com.au 10/1 Toxteth Rd Glebe NSW 2037 Australia voice: 61-2-9660-0850 homepage: http://www.maxtal.com.au/~skaller download: ftp://ftp.cs.usyd.edu/au/jskaller From gstein@lyra.org Wed Jan 26 03:26:44 2000 From: gstein@lyra.org (Greg Stein) Date: Tue, 25 Jan 2000 19:26:44 -0800 (PST) Subject: [Types-sig] experimenting with type checking (was: how to do a typecase?) In-Reply-To: <200001251040.LAA04242@python.inrialpes.fr> Message-ID: On Tue, 25 Jan 2000, Vladimir Marangozov wrote: >... > Ah. Let me plug that it's nice to see the discussion changing its tone > from "type constraining" to "type checking", although the two aspects are > tightly related to each other. With the latest proposals, I think there's > enough material already for experimenting with a basic type checker which > could perform some ERR. Ah! But the experimentation has begun... My prototype is available at http://www.lyra.org/greg/python/typesys/. I've got some changes made over the weekend to check in and re-publish. I'll send another mail at that point. But if you're in a hurry, then go grab a copy of the code now :-) Cheers, -g -- Greg Stein, http://www.lyra.org/ From gstein@lyra.org Wed Jan 26 03:32:36 2000 From: gstein@lyra.org (Greg Stein) Date: Tue, 25 Jan 2000 19:32:36 -0800 (PST) Subject: [Types-sig] Re: evaluating '!' In-Reply-To: <388D9E57.BCEE9CFC@maxtal.com.au> Message-ID: On Wed, 26 Jan 2000, skaller wrote: > scott wrote: > > ... stuff about '!' constraining the system we can build ... > > ... John responding that '!' can exist in *addition* to another system ... I was going to respond to Scott and say the same thing, but I think John said it much better. It is untrue that '!' will limit our type checking system in *any* fashion. It is a (IMO) a great addition and will definitely help any other system that we might design. To me, it is a no-brainer place to start. Adding certain forms of "decl" and certain forms of syntax for type declarators are also no brainers. I just get a bit more worried about complex type declarators, parameterization, typedef mechanisms, and interfaces. Those will take a bit more discussion to reach a solid, consensual basis on their syntax, semantics, and benefits. Cheers, -g -- Greg Stein, http://www.lyra.org/ From skaller@maxtal.com.au Wed Jan 26 10:35:42 2000 From: skaller@maxtal.com.au (skaller) Date: Wed, 26 Jan 2000 21:35:42 +1100 Subject: [Types-sig] Re: evaluating '!' References: Message-ID: <388ECDFE.57EC3B8@maxtal.com.au> Greg Stein wrote: > > On Wed, 26 Jan 2000, skaller wrote: > > scott wrote: > > > ... stuff about '!' constraining the system we can build ... > > > > ... John responding that '!' can exist in *addition* to another system ... > > I was going to respond to Scott and say the same thing, but I think John > said it much better. It is untrue that '!' will limit our type checking > system in *any* fashion. It is a (IMO) a great addition and will > definitely help any other system that we might design. To me, it is a > no-brainer place to start. Adding certain forms of "decl" and certain > forms of syntax for type declarators are also no brainers. I just get a > bit more worried about complex type declarators, parameterization, typedef > mechanisms, and interfaces. Those will take a bit more discussion to reach > a solid, consensual basis on their syntax, semantics, and benefits. I get worried by _any_ decl, for two reasons: one, it introduces another compile time statement -- I'd like to get RID of the other one (global). Secondly, it isn't clear what the semantics are/should be. So I get worried a bit before Greg .. but agree that the complexity of an extended declaration sublanguage introduces additional uncertainties. -- John (Max) Skaller, mailto:skaller@maxtal.com.au 10/1 Toxteth Rd Glebe NSW 2037 Australia voice: 61-2-9660-0850 homepage: http://www.maxtal.com.au/~skaller download: ftp://ftp.cs.usyd.edu/au/jskaller From gstein@lyra.org Wed Jan 26 15:50:49 2000 From: gstein@lyra.org (Greg Stein) Date: Wed, 26 Jan 2000 07:50:49 -0800 (PST) Subject: [Types-sig] updated prototype Message-ID: Hi all! I've updated my prototype type checker. Some new (visible) features: - A new "decl" form in the interface parser: decl intrinsic __add__: def(Any)->Any This form is used to state that the object using this interface supports addition, but NOT the __add__ attribute. These "intrinsic" methods are used by the builtin types. - Added a default search for "builtins.pi" to define interfaces for the builtin types. I've added ListType in there for now. The names for the interfaces should correspond to the names found in the "types" module. - Binary operators have "real" type checking: the arguments are checked and the result type is defined. [ sequence concatenation and repetition is incomplete ] - The interface parser now uses '|' rather than 'or' [ note: builtins.pi uses list notation (i.e. [Any]) but the parser does not see the brackets at the moment ] - "not" operator's return type is always Int My type system pages are at: http://www.lyra.org/greg/python/typesys/ A snapshot is available via that page or at: http://www.lyra.org/greg/python/typesys/typesys.tar.gz The CVS repository can be browsed at: http://www.lyra.org/cgi-bin/viewcvs/gjspy/typesys/ (the link is also available via the type system page) Comments, suggestions, criticisms, etc are well-appreciated! Cheers, -g -- Greg Stein, http://www.lyra.org/ From scott@chronis.pobox.com Wed Jan 26 18:14:46 2000 From: scott@chronis.pobox.com (scott) Date: Wed, 26 Jan 2000 13:14:46 -0500 Subject: [Types-sig] Re: evaluating '!' In-Reply-To: References: <388D9E57.BCEE9CFC@maxtal.com.au> Message-ID: <20000126131446.A86513@chronis.pobox.com> On Tue, Jan 25, 2000 at 07:32:36PM -0800, Greg Stein wrote: > On Wed, 26 Jan 2000, skaller wrote: > > scott wrote: > > > ... stuff about '!' constraining the system we can build ... > > > > ... John responding that '!' can exist in *addition* to another system ... > > I was going to respond to Scott and say the same thing, but I think John > said it much better. >It is untrue that '!' will limit our type checking > system in *any* fashion. That's not really what John said. I would paraphrase by stating that he thinks a mitigating factor of the drawback it presents is that one could add it to any system -- indeed even any system that had no need for it. Then, one could create code without even using it and, for that code, attain the qualities of a type checking system without it. But, it limits the type checking system in the following fashion: if the system has a !-like construct, and you run it's compile time checker on some random code, and you are presented with no errors, then you know *nothing* about the relative type safety of that program, because it is possible that each and every check the program makes has been forced to be correct by the !-like construct, when in fact those checks are wrong. A system without a !-like construct does not have this quality. Does that quality sound desirable to you? To me, the word undesirable is understatement in the extreme. We *can* know things about programs that pass a compile time checker without errors. Exactly what we could know is currently being discussed while we address the things that worry you below, and is algo-dependent for sure. But the more we know from a clean compile time check, the better. >It is a (IMO) a great addition and will > definitely help any other system that we might design. To me, it is a > no-brainer place to start. Adding certain forms of "decl" and certain > forms of syntax for type declarators are also no brainers. I just get a > bit more worried about complex type declarators, parameterization, typedef > mechanisms, and interfaces. Those will take a bit more discussion to reach > a solid, consensual basis on their syntax, semantics, and benefits. The no-brainer quality I see in your ! construct is not the ease of implementation, but the effect on the system. I'm happy to agree to disagree on whether we want a system with this quality, on whether you think the pros outweight this con, etc; but I can't stand by and watch !-propoganda that seems factually incorrect to me (like your statement above). scott PS - have you tried to compile your grammar for type declarators into python and run it yet? I have compiled an altered grammar into python and ran into a few issues you might want to know about: 1) inline function delcarations as you wrote the grammar cause all lambda usage to be a syntax error. 2) out of line function declarations produce hard-to-work-around ambiguities in python's parser when you have 'def' [ NAME ':' ] instead of 'def' [':' NAME ] because can be a name too. 3) adding words in hidden parts of the grammar, such as 'intrinsic', causes the word 'intrinsic' to become a keyword. The workarounds are viable, but not clean by any means and probably not very easy either. From tim_one@email.msn.com Wed Jan 26 20:46:27 2000 From: tim_one@email.msn.com (Tim Peters) Date: Wed, 26 Jan 2000 15:46:27 -0500 Subject: [Types-sig] Re: evaluating '!' In-Reply-To: <388ECDFE.57EC3B8@maxtal.com.au> Message-ID: <000701bf683e$6b9a5240$e0a2143f@tim> [John Skaller] > I get worried by _any_ decl, for two reasons: one, it > introduces another compile time statement -- I'd like to get > RID of the other one (global). > > Secondly, it isn't clear what the semantics are/should be. So > I get worried a bit before Greg .. but agree that the complexity > of an extended declaration sublanguage introduces additional > uncertainties. The inability to name what we're talking about can be seen as a benefit only if we're so confused we're afraid to say anything <0.5 wink>. The semantics are certainly vague at this point, but equally unclear for both decl and type-assert. Greg defines the latter's semantics by appeal to the undefined "consistent", and later pushes that off in turn to the also undefined (in the New World) semantics of the isinstance function. Guido's proposal gives concrete examples, but no general rules, and actually raises more questions than pretends to give answers. It's not even clear to me that, *however* enhanced, isinstance will be appropriate for defining type-checking behavior (c.f. comments from you and Guido about the (certain to you, plausible to Guido) necessity to distinguish between subtyping and subclassing). Still, if you're supernaturally comfortable with the semantics of "!" already, here are plausible semantics for decl: within the scope of decl var: typedecl each textual instance of var behaves as if (var ! typedecl} had been written instead (with obvious modifications for appearances of var as a binding target), and there are a pile of undefined yet highly attractive rules about when that will raise an error at compile-time. IOW, toss out decl, and nothing gets easier "even in theory": type-assert alone raises the same issues. At this point I think the easy stuff is behind us. I'd suggest that, to make progress, we pick someone else's brains rather than toss out our own half-baked ideas at random. Else it's Tower of Babel time now. In the absence of a better suggestion, I recommend basing further discussion on the terminology and concepts from the nice survey paper Guido referenced in his proposal: [Bruce96] Kim Bruce: Typing in object-oriented languages: Achieving expressiveness and safety. http://www.cs.williams.edu/~kim/README.html That's what I intend to do, although Real Life insists I vanish for about a week ... From scott@chronis.pobox.com Sat Jan 29 00:06:45 2000 From: scott@chronis.pobox.com (scott) Date: Fri, 28 Jan 2000 19:06:45 -0500 Subject: [Types-sig] STICK - static type and interface checking for python Message-ID: <20000128190645.A26999@chronis.pobox.com> With much help from the discussions on this list and a colleague, I've managed to come up with a type checking system for python. It's not done, by any means, but it certainly does a lot. It's the result of a few weeks of obsessively banging my head against the problem until very recently, when it became apparent that there is atleast one system which, as a whole, is quite feasible and provides for a lot of the functionality we've all wanted -- polymorphism, safety, and a framework for flexibility. Instead of rushing to complete the system, I figured it best to get this guy out there, hopefully as a means to further our discussions until we come up with a more coherent picture of what we want. With any luck, we'll be able to hammer it into what we want instead of me hammering it into what I want. Even without such luck, I really think it's worth a good look, because it is largely functional, and most of all because it approaches a single system much more so than the grab bags of mini specifications we've been considering to date. It is available at ftp://chronis.pobox.com/pub/python/stick-20000128.tgz I'm attaching the NOTES file from the project to try to peak some of your curiousities into actually downloading and playing with stick. Who, Me?'ly scott -------------------------------------------------------------------------- Type Safety STICK derives its type safety from a few guiding principles. First, any time any operation happens in the python compile tree that involves information loss about types, STICK assumes that all possible information that might be lost is in fact lost. I just call this being conservative. Second, stick has strict assignment checking. When assigning a type T to a type T1, if the union of T and T1 is not the same as T itself, the assignment is illegal. Third, the types T and T1 are always derived from the set of all types that the corresponding values may ever have throughout the life of the body of checked code. This last quality is assured partially from each of the first two guiding principles. STICK does *not* make any significant attempt to do flow analysis. The closest it comes to flow analysis is looking at modules at the point in the parse tree where the modules are imported to attempt to find a few extra potential name errors. I don't believe there's any need for stick to mark particular pieces of code with type information that is not reliable accross the entire body of code. In fact, I believe that if a type system were to rely on doing this marking of code segments in python, it would have no choice but to be either unmanageable or full of thousands of type-safety holes. While I do not have a mathematical proof of type safety, and I'm not one who is prone to doing such work, my study of type checking and understanding of python lead me to believe STICK is type safe in the sense that there are no surprising holes in the system. I could well be wrong, so if anyone can show where I am wrong it would be greatly appreciated. I am fully aware of the affects of eval, exec, __import__ and the possibility of mucking with code objects and/or byte code. These things will never be type checkable, but I think the system would be type safe if it were able to consider all of these uses illegal. Making it do that seems trivial when compared to the task of designing and implementing this system, though. Keywords This type checking implementation is keyword sparse. It introduces only two new keywords, 'datatype' and 'decl'. Along with these keywords, there are a number of operators that are put to use, such as '->' and '!'. These operators could sometimes be replaced by other keywords for the sake of clarity, but I chose to use them since introducing key words is so frowned upon in python. Whether or not these things are operators or keywords is irrelevant to this project. This project is a prototype, and replacing such symbols is trivial. What is important is clarity of semantics. See section on Grammar below. Runtime implications Since this system is polymorphic, it requires alterations to pythons runtime environment. Specifically, the runtime evironment must be able to get the type of just about any arbitrary python value for the pupose of evaluating deconstructors (see the file samples/or_test.py for more info on that). Since it is questionable whether including runtime checks that reinforce the compile time checks will cause potentially serious performance problems, and since the compile time checks seem to be full proof in the absense of exec, eval, __import__ and family, I recomend *not* having any runtime changes to python except that which is necessary to implement the deconstructors. Also, those particular problem functions and statements can be checked for and warned about. Reduplicating compile checks at run time is atleast largely if not completely redundant. Grammar The grammar for STICK python is in the file 'Grammar'. The relationship between grammars and semantics is interesting. One can always come up with a different grammar that supports identical semantics. One can also come up with a different grammar that affects the way the system works. I am fully aware of the fact that the Grammar that STICK presents will not be exactly what the python community is looking for. It is my own creation and has my own idea of 'Pythonic' written all over it. But everyone has a different idea of what's more Pythonic and what's not. Because of this, I think STICK should welcome any changes to it's grammar that seems to satisfy more people than myself, and should be subject to final say from Guido himself -- that is, if STICK turns out to be viable for python in the first place. Changes to the grammar which do not alter STICK's semantics I consider trivial. All other suggested changes would probably fair well only if they take into consideration the semantics of the system, and most crucially, how type safety is achieved in the system. Any suggested alterations to the grammar which imply a loss of type safety in any way should be weighed heavily with the benefits such a change may bring out. Type safety is important, not only as a means to early code checking, but also as a means to optimizers that can produce reliable code and documentation that is uniform, thorough and correct. Polymorphism As I understand the term polymorphism, there are a few specific meanings which all revolve around the idea that a value can be of a set of types rather than a specific type. STICK provides for polymorphism insofar as it supports a means of dealing with arbitrary OR'd types. Additionally, STICK provides a way of using something akin to what has been called parameterized types, but that I find more useful and intuitive less cluttery. I am unaware of the technical term for this, but it looks like this: # # function can take any type and will return lists of that type. The # particular type that the function takes can vary accross different function # calls. STICK currently supports this. # decl f: def(~A) -> [~A] Additionally, STICK will attempt to add a similar such feature to it's notion of interfaces. But that comes later. It is likely, but I'm not sure exactly what form it would be available in. Disambiguating Types When you have a value that could be one of two fairly distinct types, you can't do much with that value. So there is a need to disambiguate the types. This has been discussed at length on the types-sig. The general consensus has thus far been divided between using a type-checker-override operator (Greg Stein's !) and thinking of alternative ways of making grammatical constructs from other languages more pythonic. STICK uses a refreshing approach that has none of the drawbacks of either of these systems. I don't know if any other languages use something like this, but I got the idea as a sort of scaled down typecase and a sortof inverted use of constructors (a la ML), all with a bit of OOP interface. I call it 'deconstructing' types for lack of a better term. There was a discussion on the types-sig where guido said that he believed in 'structural' type checking, or atleast he thought he did ;) This placed in contrast to using constructors. STICK changes this contrast to an inversion: instead of using constructors, use deconstructors when you want to split up sets of types. This way, the type creation is structural, as it seems natural for python. Most importantly, and you don't have to do *any* flow analysis to guarantee type safety, which in the face of python's dynamism seems like a necessary starting point. The way STICK disambiguates is by making datatype objects that have OR'd types have methods which are callable to get the appropriate type or a default. For more info, see samples/or_test.py. This precludes the necessity of a grammatical construct, and wraps up all the things that a user would have to supply to a builtin method inside datatype objects. None None always requires special attention. In STICK, None has it's own distinct type. Many other systems have a None parallell that is implicitly in the set of every type: sort of defined as the empty set. This is worth consideration for STICK. This idea does prevent the type checker from catching confusion between None and any other type, but it also has it's conveniences: less need to use the type engine, making the interface to deconstructors even more easy by making the second argument optional without the fear of raising an exception at run time. Declarations One thing STICK is good at is limiting the need for declarations. Every assignable value acts as if it were declared with whatever type is first assigned. This means that 'x = 1' is equivalent to 'decl x: int; x = 1', so long as that is x's first usage. It also means that x = [ 1, 2, 3, "foo"][0] is the same as declaring x to be of type 'int | string'. Limiting declarations to variables which are instantiated with some value that is a subset of the total values for that variable makes life easy. Interfaces Interfaces aren't done in stick yet. The idea I have for them is to make interfaces distinct from classes or modules, and to allow inheritance that will create subtypes. Correspondingly, there can be a way to declare that instances of a class or a module conforms to an interface. With the very strict base of plain type checking, this shouldn't be so hard. You can guess the semantics from reading this and looking at the grammar. Feedback All feedback is most welcome, and should probably be directed to the types-sig (types-sig@python.org). I ask only one favor, which is to try to keep the posts short and addressing particular issues, even if the particular issue is with STICK overall. It's hard for everyone on the sig to read huge proposals and rants. And before Tim Peters calls STICK half-assed, I'll go ahead an say it. It's certainly half assed in terms of some coding style and some organizational issues and the degree of completion is <= 0.1-assed. But it is still the result of a huge amount of work, research, and primarily thought, on the part of everyone who has contributed to the types-sig goals of recent, including one particular Josh Marcus. Where it exceeds compared to other proposals is that the system as a whole seems relatively safe and flexible and coherent, so you can build on it well. Where it lacks is coordination and feedback and contribution from everyone out there. Hopefully this presentation will change that. Scott Cotton IC Group, Inc scott@foobox.com From jeremy@cnri.reston.va.us Sat Jan 29 00:13:07 2000 From: jeremy@cnri.reston.va.us (Jeremy Hylton) Date: Fri, 28 Jan 2000 19:13:07 -0500 (EST) Subject: [Types-sig] Typing In-Reply-To: <388DC05A.28DA4ED0@maxtal.com.au> References: <000401bf6706$9b6c6460$632d153f@tim> <388DA1F7.36C8E04@maxtal.com.au> <388DC05A.28DA4ED0@maxtal.com.au> Message-ID: <14482.12435.62487.939092@bitdiddle.cnri.reston.va.us> I want to clarify a single issue you raised in your message. >In THIS code, a simple checker will find no error, but in >THIS code: > > decl y : int | string > decl x : int | string > for i in [0,1]: > x = [10,'ten'][i] > y = [20,'twenty][i] > print x + y > >we must issue an error, because the possibility that x is an int, >and y is a string, and x + y would be wrong, exists .. and cannot >be excluded by bottom up analysis. It is not definite there >is an error .. and it is not definite that there is not. When you say "we must issue an error," I think you should say "we must insert a runtime check." I realize that Guido has as a stated goal that checked modules do not raise type errors as runtime. But as you say, it is impossible to check your example at compile time. We would lose enormous expressivity if we limited ourselves to programs that are entirely statically checkable at compile time. I think there is no way to avoid some amount of runtime checks. (Heck, Java couldn't escape them either.) If we do issue a dynamic check for the example code, we might also issue a compile time warning: "Your code might raise a TypeError at runtime: int + string." I'm not sure how helpful this warning would be; I expect it primarily depends on how often they occur. (I think I agree with your intuition that union types like these will tend to pile up into lots of hairy unions.) Jeremy From scott@chronis.pobox.com Sat Jan 29 01:15:26 2000 From: scott@chronis.pobox.com (scott) Date: Fri, 28 Jan 2000 20:15:26 -0500 Subject: [Types-sig] STICK - static type and interface checking for python In-Reply-To: <20000128190645.A26999@chronis.pobox.com> References: <20000128190645.A26999@chronis.pobox.com> Message-ID: <20000128201526.A27613@chronis.pobox.com> On Fri, Jan 28, 2000 at 07:06:45PM -0500, scott wrote: [...] > strict assignment checking. When assigning a type T to a type T1, if the > union of T and T1 is not the same as T itself, the assignment is illegal. ^ should be T1, duh scott From Moshe Zadka Sat Jan 29 08:02:44 2000 From: Moshe Zadka (Moshe Zadka) Date: Sat, 29 Jan 2000 10:02:44 +0200 (IST) Subject: [Types-sig] Re: puzzle about subtyping and parameterization In-Reply-To: <14470.4022.551614.971388@goon.cnri.reston.va.us> Message-ID: On Wed, 19 Jan 2000, Jeremy Hylton wrote: > I thought I had a solution until you mentioned strings :-(. For > numeric types, I was going to propose that number be used most of the > time, and that 1/2 == .5 and that 2**45 doesn't raise an 1/2 == 0.5 is the wrong solution to the right problem. Floating point arithmetic should *never* be thrust upon people: it's just plain hard. (Let me just note that since I actually have to work a lot of floating point arithmetic, I can see people with a lot of programmin experience, myself included, making many subtle floating point bugs) This is only marginally type-sig stuff, and is certainly not part of the specs for a type checker, but it should be the stuff for any Pythonic type system. -- Moshe Zadka . INTERNET: Learn what you know. Share what you don't. From tim_one@email.msn.com Sun Jan 30 08:00:51 2000 From: tim_one@email.msn.com (Tim Peters) Date: Sun, 30 Jan 2000 03:00:51 -0500 Subject: [Types-sig] Re: puzzle about subtyping and parameterization In-Reply-To: Message-ID: <000501bf6af8$200e3960$422d153f@tim> [posted and mailed] [Moshe Zadka] > 1/2 == 0.5 is the wrong solution to the right problem. Floating point > arithmetic should *never* be thrust upon people: it's just plain hard. > > (Let me just note that since I actually have to work a lot of floating > point arithmetic, I can see people with a lot of programmin experience, > myself included, making many subtle floating point bugs) There are no good solutions to this. Guido & I corresponded about Python's numerics before the first release, and agreed that ABC's default use of rational arithmetic was even worse for beginners. Yes, 1/49*49 works out to exacty 1 (btw, 49 is the least positive integer such that 1./i*i != 1 in properly rounded IEEE double arithmetic). But space and time use can grow at an astonishing rate when using unbounded rationals, and fighting that effectively requires numerical analysis just as sophisticated as fp requires -- it reraises all the same issues, since the only way *to* fight it effectively is to retreat to approximations from time to time. It's actually harder than fp in the end, because you have to figure how much to approximate and when, then force it all "by hand". The constructive reals don't harbor any accuracy surprises-- even (unlike rationals) in the presence of transcendental functions --but also run pig slow; and try explaining to a newbie that comparisons and conversion to int are undecidable . The best compromise I've seen is REXX's, which was designed for newbies from the start. It's *decimal* floating point (like calculators do), but with user-specifiable precision and a huge exponent range. Space and time use are bounded, and if a computation looks flaky you can just rerun it with higher precision. > This is only marginally type-sig stuff, and is certainly not part of the > specs for a type checker, but it should be the stuff for any Pythonic > type system. I bet Python2 will grow some new numeric types, and Guido has already said a bit about the numeric type system in his Types-SIG proposal. > INTERNET: Learn what you know. > Share what you don't. I love that sig! Did you make it up, Moshe? so-true-it-hurts-ly y'rs - tim From tim_one@email.msn.com Sun Jan 30 08:49:17 2000 From: tim_one@email.msn.com (Tim Peters) Date: Sun, 30 Jan 2000 03:49:17 -0500 Subject: [Types-sig] Typing In-Reply-To: <14482.12435.62487.939092@bitdiddle.cnri.reston.va.us> Message-ID: <000801bf6afe$e4756700$422d153f@tim> [John Skaller] > decl y : int | string > decl x : int | string > for i in [0,1]: > x = [10,'ten'][i] > y = [20,'twenty][i] > print x + y > > we must issue an error, because the possibility that x is an int, > and y is a string, and x + y would be wrong, exists .. and cannot > be excluded by bottom up analysis. It is not definite there > is an error .. and it is not definite that there is not. [Jeremy Hylton] > When you say "we must issue an error," I think you should say "we must > insert a runtime check." I realize that Guido has as a stated goal > that checked modules do not raise type errors as runtime. But as you > say, it is impossible to check your example at compile time. Just repeating that there are three kinds of people and so three kinds of desirable behavior: compile-time error; compile-time warning + runtime check; compile-time warning but no runtime check; and couldn't care less. It occurs to me that that's four behaviors, so some pair of them must be identical despite appearances . > We would lose enormous expressivity if we limited ourselves to > programs that are entirely statically checkable at compile time. Which is exactly what some people *want*, and they have legitimate reasons (both OPT and ERR) for so wanting. Don't screw 'em "on principle" -- their principles differ. This has got to be a compile option. > I think there is no way to avoid some amount of runtime checks. (Heck, > Java couldn't escape them either.) This is a non-problem if we don't force everyone into the same mold. For example, one plausible use of typed Python is as an approximation to Martijn's Swallow project: OPT via full static typing, and if the compiler can't achieve that for some piece of code, that's fine! It should just raise an error. The OPT-starved author will be delighted to alter their code until the compiler *can* deal with all of it. That's what they want to do -- it's their whole *point* in using typing at all. You can be sure that some companies will want the same behavior, but for ERR reasons in "critical core subsystems". Most of the rest of us will indeed want to compromise most of the time, and accept some runtime checks. Nobody is "wrong" here. > If we do issue a dynamic check for the example code, we might also > issue a compile time warning: "Your code might raise a TypeError at > runtime: int + string." I'm not sure how helpful this warning would > be; I expect it primarily depends on how often they occur. Which is why this axis should be option-controllable too. > (I think I agree with your intuition that union types like these will > tend to pile up into lots of hairy unions.) Oh ya! They do. "One name, one type" will work best, in Python as in every other language . catering-to-all-possible-behavior-doesn't-mean-liking-it-too-ly y'rs - tim From paul@prescod.net Mon Jan 31 09:07:52 2000 From: paul@prescod.net (Paul Prescod) Date: Mon, 31 Jan 2000 03:07:52 -0600 Subject: [Types-sig] Whereto from here? Message-ID: <389550E8.C4D52900@prescod.net> First a recap and then a proposal. Recap: Guido presented his ideas (based on our ideas) to the assembled IPC8 group. He got somewhat mixed reviews. I expected the mixed reviews based on the community's distrust of radical change in general and static type checking in specific.[1] Guido made it clear that we would be proceeding cautiously and slowly. This stuff is not going into Python 1.6 and quite likely not into 1.7. On the other hand, static type checking will likely be THE major feature of Python 3000 (which will likely be renamed "2" when the time comes). The long and short is that we have many months, probably a year before we have to produce something solid. The sooner we do, the sooner we can start testing, of course. But we should not rush our design -- there is a lot riding on us getting it right (which means coming up with a system that Python programmers can learn to love). As far as I am concerned, we met our first deadline. We wanted to have something to show at IPC8 and Guido showed something. QED. No, we did not have code that implemented his proposal. No, his proposal was not even done. Those were unrealistic goals, set before we understood the scope of the problem (sociologically and technically speaking). We accomplished something useful in our first month and a half and now we have a much longer time span to meet all of our goals. This delayed time-schedule fits me fine because am one of a seemingly small minority that believes that there are an incredible number of hairy details in the corners that need to be documented in mind numbingly precise detail so that there can be multiple, indepenent, interoperable implementations. My last proposal is probably half the size it needs to be. The more informal proposals that others have batted around are also not close yet. We are developing something as sophisticated as, say, XML which was my last big standardization effort. It took two years with dozens of posts per year. Okay, so one argument for slowing down is allowing the spec-writer to keep up with the discussion. I would be totally relieved if someone else were to step up and agree to be the spec writer. I would need to trust the person to really be committed to a tight spec based on community concensus (and I know that mine is far from ideal on both points) and Guido would have to agree. The second argument for a slow down is that we "professional types-sig posters" have lost the community. We post too much too quickly. When we had an IPC8 deadline this was a requirement. Now we have to separate issues. We need to let people know what issues we are discussing. We need to invite interested parties into our discussions. I think that if we slow down we can achieve a more democractic process. The third argument and final argument for slowing down is that Python 1.6 is coming out and some of us have responsibilities relating to it. I want to do work on XML stuff. Greg wants to do import and distribution stuff. Guido has to, well, Guido has to do Python 1.6. It makes no sense for us to invent a system behind Guido's back. Unlike the US Congress, we have no practical mechanism for overriding his veto. :) I think our first goal should be to figure out which issues we positively agree on so that we do not have to backtrack and issues that are way up in the air so that we know what is still to be decided. Let's say I get a list of those things out by Wednesday for one week discussion. Then I will make a list of unresolved issues and we can start going through them. Over time we will add to the list but the unresolved issues should migrate from being Big Ideas like "runtime or dynamic", "are interfaces classes" and so forth to being Small and Detailed: "colon or as", "int, integer or integral". Opinions? Paul Prescod [1]Some days I feel nervous myself. But most days I believe strongly that if it is possible to combine optional static type checking in a language with Python's other virtues, someone will do it and that someone will replace all languages in the space currently occupied by TCL, Perl, Python and Java. IMHO, the strongest arguments for Perl and Java are performance and (in the case of Java) support for "programming in the large." [2]Perhaps Python 3000 could live alongside Python 1.6/1.7. Then people can move over at their own pace and if we have destroyed the language then our variant branch(es) could quietly disappear and some other variant could take on the number "2". -- Paul Prescod - ISOGEN Consulting Engineer speaking for himself The new revolutionaries believe the time has come for an aggressive move against our oppressors. We have established a solid beachhead on Friday. We now intend to fight vigorously for 'casual Thursdays.' -- who says America's revolutionary spirit is dead? From scott@chronis.pobox.com Mon Jan 31 10:27:05 2000 From: scott@chronis.pobox.com (scott) Date: Mon, 31 Jan 2000 05:27:05 -0500 Subject: [Types-sig] Whereto from here? In-Reply-To: <389550E8.C4D52900@prescod.net> References: <389550E8.C4D52900@prescod.net> Message-ID: <20000131052705.A40424@chronis.pobox.com> On Mon, Jan 31, 2000 at 03:07:52AM -0600, Paul Prescod wrote: > First a recap and then a proposal. > [ Recap ] > > > I think our first goal should be to figure out which issues we > positively agree on so that we do not have to backtrack and issues that > are way up in the air so that we know what is still to be decided. Let's > say I get a list of those things out by Wednesday for one week > discussion. Then I will make a list of unresolved issues and we can > start going through them. Over time we will add to the list but the > unresolved issues should migrate from being Big Ideas like "runtime or > dynamic", "are interfaces classes" and so forth to being Small and > Detailed: "colon or as", "int, integer or integral". > > Opinions? I agree with most everything you have said. I would rephrase slowing down so that it does not imply working less, just that we're in phase 0.1 -- deciding what we want and if it's possible. That doesn't even put us on a timeline for any rational time estimate in the first place :) RE: small hairy details: yes there are lots of them. What's worse are the big hairy ones, though ;) we gotta decide what we want. RE losing the community: a centralized set of resources would help a lot more than us slowing down down would, IMO. I also think that the innards of a typing system are beyond what most potential users really want to think about. There needs to be a forum for discussing that stuff as well, because that will clearly have to be addressed. One of the drawbacks of types-sig is that it attempts to be a forum for both, so posts addressing technical typing issues seem to become watered down just enough to produce undue confusion. RE proposal: it is evident that the community of folks contributing to all this have different goals. I think that discussions about issues in particular would be much more fruitful if these goals were first stated, and if someone could make some decisions about which ones to keep, which ones to toss, which ones are absolutely necessary, which ones are optional, and generally how to prioritize them for the purpose of the future of python. With a mission statement that goes beyond "ERR OPT DOC", we'll likely be able to come up with a good set of requirements for a starting point of a typing and interface system. Once that happens, half the battle and 0.1% of the final coding and specification are done. The hardest part sociologically is the mission statement, and conceptually is that specification for a starting point. scott > > Paul Prescod > > [1]Some days I feel nervous myself. But most days I believe strongly > that if it is possible to combine optional static type checking in a > language with Python's other virtues, someone will do it and that > someone will replace all languages in the space currently occupied by > TCL, Perl, Python and Java. IMHO, the strongest arguments for Perl and > Java are performance and (in the case of Java) support for "programming > in the large." > > [2]Perhaps Python 3000 could live alongside Python 1.6/1.7. Then people > can move over at their own pace and if we have destroyed the language > then our variant branch(es) could quietly disappear and some other > variant could take on the number "2". > I'm over the some days feeling nervous part. A real typing system for python is not only theoretically possible, but actually doable. A real interface system can be built on top of that. Integrating type checked and not type checked code will however, not be seemless -- it's not possible to have type checked code run optimized by unchecked code without either ugly core dumps or godawfully slow runtime type checks that would normally happen at compile time, IMO. There are a set of compromises which might make such integration good enough, though. > > -- > Paul Prescod - ISOGEN Consulting Engineer speaking for himself > The new revolutionaries believe the time has come for an aggressive > move against our oppressors. We have established a solid beachhead > on Friday. We now intend to fight vigorously for 'casual Thursdays.' > -- who says America's revolutionary spirit is dead? > > _______________________________________________ > Types-SIG mailing list > Types-SIG@python.org > http://www.python.org/mailman/listinfo/types-sig From guido@CNRI.Reston.VA.US Mon Jan 31 14:17:45 2000 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Mon, 31 Jan 2000 09:17:45 -0500 Subject: [Types-sig] Whereto from here? In-Reply-To: Your message of "Mon, 31 Jan 2000 03:07:52 CST." <389550E8.C4D52900@prescod.net> References: <389550E8.C4D52900@prescod.net> Message-ID: <200001311417.JAA15113@eric.cnri.reston.va.us> Thanks for a great recap, Paul. Two points: - The point of community involvement is not to have a more "democratic" process (as in majority rules). We want the best design, not necessarily the design that gets the most votes. :-) - For those who weren't there, I want to explain Paul's reference to Python 1.6 and 1.7. We decided to go for an accellerated 1.6 release so that e.g. Unicode (which is close to being releasable) can be released before the summer. There will then be a 1.7 release later, containing the more esoteric stuff originally planned for 1.6 that hasn't been coded yet. I'll post more about this in a more general forum, so please don't write me for more details. --Guido van Rossum (home page: http://www.python.org/~guido/) From skaller@maxtal.com.au Mon Jan 31 17:46:21 2000 From: skaller@maxtal.com.au (skaller) Date: Tue, 01 Feb 2000 04:46:21 +1100 Subject: [Types-sig] Typing References: <000401bf6706$9b6c6460$632d153f@tim> <388DA1F7.36C8E04@maxtal.com.au> <388DC05A.28DA4ED0@maxtal.com.au> <14482.12435.62487.939092@bitdiddle.cnri.reston.va.us> Message-ID: <3895CA6D.A5B52EFB@maxtal.com.au> Jeremy Hylton wrote: > > I want to clarify a single issue you raised in your message. > > >In THIS code, a simple checker will find no error, but in > >THIS code: > > > > decl y : int | string > > decl x : int | string > > for i in [0,1]: > > x = [10,'ten'][i] > > y = [20,'twenty][i] > > print x + y > > > >we must issue an error, because the possibility that x is an int, > >and y is a string, and x + y would be wrong, exists .. and cannot > >be excluded by bottom up analysis. It is not definite there > >is an error .. and it is not definite that there is not. > > When you say "we must issue an error," I think you should say "we must > insert a runtime check." That's not what I meant; what I meant was that a checker required to verify correctness must issue a compile time error and _refuse_ to compile the program, NOT because it is wrong, but because it cannot be sure that the code is correct (type safe). This behaviour would be a consequence of requiring acceptance of a program guarrantee type safety at compile time (together with an inadequate checking algorithm). > I realize that Guido has as a stated goal > that checked modules do not raise type errors as runtime. But as you > say, it is impossible to check your example at compile time. And therefore, to meet the goal, the program must be rejected as possibly wrong .. even though it isn't. This is not uncommon: C89 rejects certain pointer conversions simply because the committee was not aware that they were safe. Similarly, casts are required to prevent static checkers getting upset in C and C++, even when the code would be dynamically correct. > We would lose enormous expressivity if we limited ourselves to > programs that are entirely statically checkable at compile time. That's an understatement, if by 'statically checkable' you mean 'analysis can determine the program is correct' :-) -- John (Max) Skaller, mailto:skaller@maxtal.com.au 10/1 Toxteth Rd Glebe NSW 2037 Australia voice: 61-2-9660-0850 homepage: http://www.maxtal.com.au/~skaller download: ftp://ftp.cs.usyd.edu/au/jskaller