From bwarsaw@cnri.reston.va.us (Barry A. Warsaw)  Tue Dec  1 03:44:59 1998
From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw)
Date: Mon, 30 Nov 1998 22:44:59 -0500 (EST)
Subject: [Types-sig] Re: PRE-PROPOSAL: Python Interfaces
References: <13915.20041.23787.219850@anthem.cnri.reston.va.us>
 <000501be18c6$a0bbb540$cb9e2299@tim>
Message-ID: <13923.26171.828432.697396@anthem.cnri.reston.va.us>

>>>>> "TP" == Tim Peters <tim_one@email.msn.com> writes:

    TP> 7/16'ths of me too, but I believe the groundrules for the
    TP> Scarecrow Proposal include "no language changes".  That leaves
    TP> manual, or at best semi-automated, convention for now.  In a
    TP> Python2 world, I doubt it would be more of an insanely
    TP> protracted battle than usual <wink> to get a "defer" keyword
    TP> added, provided it worked out in practice by hand first.

    TP> Given "def push(self, thing): defer" we could peek at the
    TP> bytecode today, note the senseless LOAD_GLOBAL of "defer", and
    TP> simply rewrite the user's code for them <cackle>.

An easy change for Python 1.6 would be to add something like
NotImplementedError to exceptions.py and a defer() builtin function.
The latter has some potential for existing code breakage, but since
user definitions of that attribute will silently override the builtin
one, it's probably minimal.  That might get us far enough along for
1.6 to play while leaving open a `defer' keyword later.

So trivial to implement, I've attached a patch set (against my hacked
1.5.2 -- watch for fuzz factor).  Extremely minimally tested.  Here's
what it looks like:

-------------------- snip snip --------------------
Python 1.5.2b1 (#101, Nov 30 1998, 22:33:15) [C] on sunos5
Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
>>> defer()
Traceback (innermost last):
  File "<stdin>", line 1, in ?
NotImplementedError: Implementation expected in derived class
>>> defer(1)
Traceback (innermost last):
  File "<stdin>", line 1, in ?
TypeError: function requires exactly 0 arguments; 1 given
>>> class A:
... 	def doit(self): defer()
... 
>>> class B(A):
... 	def doit(self): print 'done!'
... 
>>> a = A()
>>> b = B()
>>> a.doit()
Traceback (innermost last):
  File "<stdin>", line 1, in ?
  File "<stdin>", line 2, in doit
NotImplementedError: Implementation expected in derived class
>>> b.doit()
done!
>>> try: a.doit()
... except NotImplementedError: pass
... 
>>> 
-------------------- snip snip --------------------
Index: bltinmodule.c
===================================================================
RCS file: /projects/cvsroot/python/dist/src/Python/bltinmodule.c,v
retrieving revision 2.129
diff -c -r2.129 bltinmodule.c
*** bltinmodule.c	1998/10/01 15:33:12	2.129
--- bltinmodule.c	1998/12/01 03:32:47
***************
*** 981,986 ****
--- 963,988 ----
  
  
  static PyObject *
+ builtin_defer(self, args)
+ 	PyObject *self;
+ 	PyObject *args;
+ {
+ 	if (!PyArg_ParseTuple(args, ""))
+ 		return NULL;
+ 	PyErr_SetString(PyExc_NotImplementedError,
+ 			"Implementation expected in derived class");
+ 	return NULL;
+ }
+ 
+ 
+ static char defer_doc[] =
+ "defer()\n\
+ \n\
+ Raise a NotImplementedError unconditionally.  This is used in the body\n\
+ of `abstract' method definitions.";
+ 
+ 
+ static PyObject *
  builtin_hash(self, args)
  	PyObject *self;
  	PyObject *args;
***************
*** 1887,1892 ****
--- 1931,1937 ----
  #ifndef WITHOUT_COMPLEX
  	{"complex",	builtin_complex, 1, complex_doc},
  #endif
+ 	{"defer",       builtin_defer, 1, defer_doc},
  	{"delattr",	builtin_delattr, 1, delattr_doc},
  	{"dir",		builtin_dir, 1, dir_doc},
  	{"divmod",	builtin_divmod, 1, divmod_doc},
***************
*** 1954,1959 ****
--- 1999,2005 ----
  PyObject *PyExc_NameError;
  PyObject *PyExc_OverflowError;
  PyObject *PyExc_RuntimeError;
+ PyObject *PyExc_NotImplementedError;
  PyObject *PyExc_SyntaxError;
  PyObject *PyExc_SystemError;
  PyObject *PyExc_SystemExit;
***************
*** 1989,1994 ****
--- 2035,2041 ----
  	{"NameError",          &PyExc_NameError,          1},
  	{"OverflowError",      &PyExc_OverflowError,      1},
  	{"RuntimeError",       &PyExc_RuntimeError,       1},
+ 	{"NotImplementedError",&PyExc_NotImplementedError,1},
  	{"SyntaxError",        &PyExc_SyntaxError,        1},
  	{"SystemError",        &PyExc_SystemError,        1},
  	{"SystemExit",         &PyExc_SystemExit,         1},
Index: exceptions.py
===================================================================
RCS file: /projects/cvsroot/python/dist/src/Lib/exceptions.py,v
retrieving revision 1.11
diff -c -r1.11 exceptions.py
*** exceptions.py	1998/09/25 22:43:21	1.11
--- exceptions.py	1998/12/01 03:26:46
***************
*** 33,38 ****
--- 33,41 ----
        |
        +-- EOFError
        +-- RuntimeError
+       |    |
+       |    +-- NotImplementedError
+       |
        +-- NameError
        +-- AttributeError
        +-- SyntaxError
***************
*** 128,133 ****
--- 131,139 ----
      pass
  
  class RuntimeError(StandardError):
+     pass
+ 
+ class NotImplementedError(RuntimeError):
      pass
  
  class SystemError(StandardError):
Index: pyerrors.h
===================================================================
RCS file: /projects/cvsroot/python/dist/src/Include/pyerrors.h,v
retrieving revision 2.30
diff -c -r2.30 pyerrors.h
*** pyerrors.h	1998/07/23 15:57:34	2.30
--- pyerrors.h	1998/12/01 03:32:26
***************
*** 73,78 ****
--- 73,79 ----
  extern DL_IMPORT(PyObject *) PyExc_NameError;
  extern DL_IMPORT(PyObject *) PyExc_OverflowError;
  extern DL_IMPORT(PyObject *) PyExc_RuntimeError;
+ extern DL_IMPORT(PyObject *) PyExc_NotImplementedError;
  extern DL_IMPORT(PyObject *) PyExc_SyntaxError;
  extern DL_IMPORT(PyObject *) PyExc_SystemError;
  extern DL_IMPORT(PyObject *) PyExc_SystemExit;


From bwarsaw@cnri.reston.va.us (Barry A. Warsaw)  Tue Dec  1 04:12:51 1998
From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw)
Date: Mon, 30 Nov 1998 23:12:51 -0500 (EST)
Subject: [Types-sig] Re: Meta-classes discussion starter
References: <208801be1c9c$bbdfe100$2d232cd1@linwin.token.hapenney.com>
 <l03102801b288bbd9020d@[193.78.237.175]>
Message-ID: <13923.27843.655430.863008@anthem.cnri.reston.va.us>

>>>>> "JvR" == Just van Rossum <just@letterror.com> writes:

    JvR> That could make for some interesting Python 2.0 syntax: class
    JvR> MyClass = MyMetaClass(BaseClass1, ..., BaseClassN): ...etc.
    JvR> where class MyClass(BaseClass1, ..., BaseClassN): ...  would
    JvR> be a shortcut for class MyClass = ClassClass(BaseClass1, ...,
    JvR> BaseClassN): ...etc.  (where ClassClass is the default
    JvR> metaclass)

I haven't seen any comments on my posted syntax proposal; perhaps it's
gross, but anyway I liked it.  Or maybe because it's buried in a long
egroups posting:

    <http://www.egroups.com/list/python-classes/54.html>

Merging with Just's posting, the jist is: if `class' were short hand
for calling ClassClass, and `class' were an object you could put
attributes on, then you could stick metaclasses on the `class' object
and use them in a class definition.  I called the metaclass a `nature' 
so here's how you would create a new metaclass/nature, and then create 
a class using that new nature:

class BeanNature(class):
    pass

class.BeanNature = BeanNature

class.BeanNature BeanieBaby:
    pass

Read near the bottom of the above referenced article for details.

    JvR> 	If the __dict__ of the class that is being built has a
    JvR> __class__ key, call that thing with (name, bases, methods)

    JvR> Which means you specify a metaclass like this:

    | class MyClass:
    | 	__class__ = MyMetaClass

    JvR> Not exactly pretty, but at least it doesn't look like
    JvR> inheritance. It also means that if you want to subclass from
    JvR> MyMetaClass you write

Not exactly ugly either!  Very similar to my proposal except 1) I
chose __nature__ as the magic attribute, and 2) I didn't actually
implement it like you did, just sort of modeled it in Python.

    JvR> This is not theory: I've got it working right here, and I
    JvR> seriously propose it for 1.6, if not 1.5.2. (If anyone wants
    JvR> to play, I could post a patch.)

Yes, please do!

-Barry


From tim_one@email.msn.com  Tue Dec  1 04:30:09 1998
From: tim_one@email.msn.com (Tim Peters)
Date: Mon, 30 Nov 1998 23:30:09 -0500
Subject: [Types-sig] Re: Types-SIG digest, Vol 1 #13 - 2 msgs
In-Reply-To: <366263F4.31FAD263@aethos.co.uk>
Message-ID: <000401be1ce3$47729c80$9b9e2299@tim>

[Dominic Binks]
> ...
> I would hazard a guess that may be as much as 90% of Python code
> is regularly type following.

I'd bet that's on the high side of the truth.  From the Walker/Griswold
paper I  posted a link to over the weekend,

    An empirical study using the type inference system described here
    was conducted on a large number of Icon programs written by many
    different authors for a wide variety of applications. The results,
    which are conservative, show a range of type consistency from about
    60 to 100%, with an average of about 80%. That is, on the average,
    the operands of about 80% of the operators in these programs always
    have the same type.

It's likely lower in Python, because (I believe) Python programmers write
more polymorphic functions than do Icon programmers.

But while that's of great interest to a type inferencing system, it's of no
particular interest to those seeking static declarations:  if you want to
use declarations, you'll have to write 100% type-consistent code whenever
declared entities are involved -- and known to be type-consistent at
compile-time.  I.e., it's a tradeoff (as if that weren't apparent from the
first instant <wink>).

> ...
> There are lots of parts of Python that pose problems for type systems.
> The following example is just one (that I don't think has been given
> before, but I may be wrong in that)
>
> 	def f(x):
> 		if type(x) == StringType:
> 			...
> 		elif type(x) == IntType:
> 			...
>
> Then x can legitimately take either an Int or a String, which is
> possibly quite hard to code in a type system.

You can view it as a union of basic types, or as an atomic Strint type --
keping track of either view is easy.  If you try to pass one of those beasts
to something declared as requiring an int, you should die at compile time.
That is, there's no problem here, except for whiners insisting they need to
have it both ways at once.  I ridicule them in advance so there will be no
need for unpleasantness later <wink>.

they-can-declare-everything-"object"-ly y'rs  - tim




From scott@chronis.pobox.com  Tue Dec  1 04:37:03 1998
From: scott@chronis.pobox.com (Scott)
Date: Mon, 30 Nov 1998 23:37:03 -0500
Subject: [Types-sig] Re: Types-SIG digest, Vol 1 #13 - 2 msgs
In-Reply-To: <000401be1ce3$47729c80$9b9e2299@tim>; from Tim Peters on Mon, Nov 30, 1998 at 11:30:09PM -0500
References: <366263F4.31FAD263@aethos.co.uk> <000401be1ce3$47729c80$9b9e2299@tim>
Message-ID: <19981130233703.14646@chronis.icgroup.com>

On Mon, Nov 30, 1998 at 11:30:09PM -0500, Tim Peters wrote:
| But while that's of great interest to a type inferencing system, it's of no
| particular interest to those seeking static declarations:  if you want to
| use declarations, you'll have to write 100% type-consistent code whenever
| declared entities are involved -- and known to be type-consistent at
| compile-time.  I.e., it's a tradeoff (as if that weren't apparent from the
| first instant <wink>).


Do you think that "whenever declared entities are involved" means that
typed python will not be able to exist with untyped python in any app? 

I would hope that we could limit the type declarations to a single
module, and let untyped python use that module.  Otherwise, it seems
like the entire standard lib will have to exist in 2 forms, typed and
not. 

scott



From skaller@maxtal.com.au  Tue Dec  1 06:50:11 1998
From: skaller@maxtal.com.au (John Skaller)
Date: Tue, 01 Dec 1998 16:50:11 +1000
Subject: [Types-sig] Re: Meta-classes discussion starter
Message-ID: <1.5.4.32.19981201065011.00f5cef8@triode.net.au>

I admit great confusion at all this meta-class stuff.
I don't see it as any kind of realistic meta-programming.

Here's why. Consider a single type 'object' onto which we
can attach attributes or functions intended to be methods.

        class object: pass

already has the first property, I'm not sure about the 'new'
module, but it isn't hard to implement something (in C) that keeps
a dictionary of functions, and converts them to bound methods
on lookup. [binding the first argument is a special case of Currying
which is a special case of the lambda function .. which therefore
subsumes the notion of a bound method]

        Now consider a factory function which makes these
objects. What is it? It is just a constructor like an __init__
method, except that it also attaches methods to the object.

        And that is it. :-) There is no such thing as a class :-))
It is nothing more than a function which makes objects.

        Bases are accounted for by simply
calling 'base' factory functions before calling
more derived ones: any 'polymorphism' is nothing more
than replacement.

        The point? meta-programming has nothing to
do with classes, because they don't exist. Only functions
are of any consequence.

        And what is required for meta-programming is now
easy to see: it is the ability to (dynamically) generate
these factory functions.

        Currently, Python has very crude meta-programming
support: the exec statement. Something more structured
than generating strings and compiling them is desirable.
But what is needed is not meta-classes, but built-in primitives
designed to facilitate dynamic construction of functions.
For example, a primitive to compose two functions (in series),
[called composition], to 'compose' them in parallel [called
product formation], and to compose them as a multiplexor 
or switch [called a union or sum].

        It can be shown that these constructions,
together with a few functors like delta(x) = (x,x),
are enough to construct all functions [given a library
of primitives to start with].

        If we had these kinds of constructions in python,
we could build functions which generate functions more
easily than we can today [using exec, or by emulating
these things with classes]. And then we would have
a better meta-programming system.

        I'm not against 'encapsulating' these meta-functions
in 'meta-classes', I'm just pointing out that classes aren't
fundamental, at least in Python. 

        Of course ... once you realise all this, the distinction
between 'a function' and 'a function that produces an object'
and 'a function that produces an object which is a function' ...
etc just falls away because all functions produce objects.

        So there is only one kind of entity required.
Not an infinite regress of classes, meta-classes,
meta-meta-classes, etc.

        In fact, one can do better -- and dispense with
objects altogether: they're just a special case of functions. :-)

        Or, you can turn that around, and consider functions
as a special kind of object: and my very point is that we need
primitives to operate on this kind of object and we don't
have them.
-------------------------------------------------------
John Skaller    email: skaller@maxtal.com.au
		http://www.maxtal.com.au/~skaller
		phone: 61-2-96600850
		snail: 10/1 Toxteth Rd, Glebe NSW 2037, Australia




From tim_one@email.msn.com  Tue Dec  1 10:10:58 1998
From: tim_one@email.msn.com (Tim Peters)
Date: Tue, 1 Dec 1998 05:10:58 -0500
Subject: [Types-sig] Re: Meta-classes discussion starter
In-Reply-To: <1.5.4.32.19981201065011.00f5cef8@triode.net.au>
Message-ID: <000e01be1d12$e40e1ea0$9b9e2299@tim>

[John Skaller, deconstructs classes and finds ... no, I won't
 spoil the surprise <wink>]

> ...
> And that is it. :-) There is no such thing as a class :-))
> It is nothing more than a function which makes objects.
> ...

John, Python is not a functional language, and there's not a shadow of a
hint that it will ever become one.

I would welcome a few more carefully chosen functional features myself, but
"everything's a function" is so obviously not in Python's worldview that
this crusade has about as much chance of success as a "and how do we express
functions? with a character set, of course! so only strings are of any
consequence, and what is required for meta-programming is richer builtin
string synthesis facilities" rant.  At least try to sell it as "everything's
a callable object!" so a few may be tricked into thinking it's Python <0.9
wink>.

> ..
> If we had these kinds of constructions in python, we could build
> functions which generate functions more easily than we can today
> [using exec, or by emulating these things with classes]. And then
> we would have a better meta-programming system.

What we would have is another functional language, a niche already fatally
over-served by others.  They're elegant!  Succinct!  Powerful!  Gorgeous!
Virtually unused!  Most people find them hostile and austere.  The repeated
failure of functional languages to catch on over 25+ glorious years is hard
to wish away.  You would like them, though!  I sure did (but I don't often
confuse them with Python, and when I do Guido ignores me until I snap out of
it ...).

> ...
>         I'm not against 'encapsulating' these meta-functions
> in 'meta-classes', I'm just pointing out that classes aren't
> fundamental, at least in Python.

I agree that they're not, today, but even less so are functions today; part
of the thrust of "healing the type/class split" appears to be to *make*
classes fundamental to the language, though.  Good move.  People like
classes, every  anthropomorphous little bit of 'em (children! parents!
birth! death! history, memory, "jobs" their instances are even "contracted"
to do! it's so *cute* -- helpful, too).

>         Or, you can turn that around, and consider functions
> as a special kind of object:

That's been Python's worldview from the start.

> and my very point is that we need primitives to operate on this
> kind of object and we don't have them.

How about a specific proposal?  You've made the high-concept "series and
parallel and switch" pitch before, but until there's some meat on its bones
it will most likely get tossed to the dogs <woof>.

gimmicks-are-an-easier-sell-than-philosophies-which-is-why-you-
    never-see-ronco's-amazing-new-ontological-clarifier-on-wee-
    hour-american-tv-ly y'rs  - tim




From tim_one@email.msn.com  Tue Dec  1 10:11:00 1998
From: tim_one@email.msn.com (Tim Peters)
Date: Tue, 1 Dec 1998 05:11:00 -0500
Subject: [Types-sig] Re: Types-SIG digest, Vol 1 #13 - 2 msgs
In-Reply-To: <19981130233703.14646@chronis.icgroup.com>
Message-ID: <000f01be1d12$e5763a20$9b9e2299@tim>

> ...
> if you want to use declarations, you'll have to write 100% type-
> consistent code whenever declared entities are involved -- and known
> to be type-consistent at compile-time.

[scott@chronis.pobox.com]
> Do you think that "whenever declared entities are involved" means
> that typed python will not be able to exist with untyped python
> in any app?

Oh no, not at all.  There's nothing wrong in mixing things, and some of the
*goals* people have in mind likely require mixing 'em.  As a dumb but
suggestive pseudo-example, the ": Int" here

num: Int = 0
for thing in sequence:
    if predicate(thing):
        num = num + 1

could be used as a hint to the compiler that "num" needn't support the
runtime burden of a full reference-counted int-in-a-box implementation,
while saying nothing about the types of thing, sequence or predicate.
People will eventually get around to demanding an "everything in this module
*must* be declared!" pragma too, of course.

Until the details are fleshed out, it's hard to guess whether it will be
more pain or gain (if it were obvious now, 1/3rd of this SIG wouldn't be
needed).

> I would hope that we could limit the type declarations to a single
> module, and let untyped python use that module.  Otherwise, it seems
> like the entire standard lib will have to exist in 2 forms, typed and
> not

That needn't be:  if the "most general" type is (say) called Object, then in
an optional static-typing world anything that isn't explicitly typed is
implicitly typed as Object.  And anything can be passed where an Object is
expected, so untyped functions can be passed anything (explicitly typed or
not).  Passing an Int where an Object is expected is fine; the other way
around is not.  So the whole std library works without change, although it
would be naive to believe there wouldn't be increasing pressure to spin off
typed variants (e.g., a typed variant of "math" that accepted only
declared-to-be floats, so it could skip the runtime "is this thing a float
or what?" checks).

People who have used C++ templates extensively will have a feel for what an
enormous amount of code bloat this can lead to (Stroustrup often recommends
that templates do casted inline redirection to a void*-slinging base class
to avoid that).  It's not *all* roses and bleeding gums <wink>.

confusion-is-healthy-ly y'rs  - tim




From just@letterror.com  Tue Dec  1 11:16:04 1998
From: just@letterror.com (Just van Rossum)
Date: Tue, 1 Dec 1998 12:16:04 +0100
Subject: [Types-sig] Re: Meta-classes discussion starter
In-Reply-To: <13923.27843.655430.863008@anthem.cnri.reston.va.us>
References: <208801be1c9c$bbdfe100$2d232cd1@linwin.token.hapenney.com>
 <l03102801b288bbd9020d@[193.78.237.175]>
Message-ID: <l03102802b2897d984c09@[193.78.237.139]>

At 11:12 PM -0500 11/30/98, Barry A. Warsaw wrote:
>I haven't seen any comments on my posted syntax proposal; perhaps it's
>gross, but anyway I liked it.  Or maybe because it's buried in a long
>egroups posting:
>
>    <http://www.egroups.com/list/python-classes/54.html>

(Hm, since your post contains mostly code, and egroups screws it up
majorly, I cannot read it... If you still have it, I'd sure appreciate it
as prvt mail.)

>    JvR> Which means you specify a metaclass like this:
>
>    | class MyClass:
>    | 	__class__ = MyMetaClass
>
>    JvR> Not exactly pretty, but at least it doesn't look like
>    JvR> inheritance. It also means that if you want to subclass from
>    JvR> MyMetaClass you write
>
>Not exactly ugly either!  Very similar to my proposal except 1) I
>chose __nature__ as the magic attribute, and 2) I didn't actually
>implement it like you did, just sort of modeled it in Python.

I initially chose __class__ just because Guido's hook uses that, too. Turns
out that because of that the two Brotherly Hooks complement each other
wonderfully!

>    JvR> (If anyone wants
>    JvR> to play, I could post a patch.)
>
>Yes, please do!

See below... It's a against the current state of Python 1.5.2b1. It should
be 100% compatible with the existing hooks (ie. I tried carefully not to
break things).

Just


- - - - - 8< - - - - - - - - - - - - -
*** ceval.c     Tue Dec  1 05:44:30 1998
--- jmetapatch/ceval.c  Tue Dec  1 05:44:12 1998
***************
*** 2672,2681 ****
--- 2672,2738 ----
        if (!PyString_Check(name)) {
                PyErr_SetString(PyExc_SystemError,
                                "build_class witn non-string name");
                return NULL;
        }
+       /*      __BEGIN__ of Just's Hook
+
+               Guido's metahook is defined as follows:
+                       - if one of the bases has a __class__ attribute (but is
+                         itself not a class!), call that thing with (name,
+                         bases, methods)
+               In addition I propose almost the opposite:
+                       - if the "methods" dict (__dict__ from the Python
+                         perspective) has a __class__ key, call that thing
with
+                         (name, bases, methods)
+
+               This means that metaclasses are not special anymore, and
+               you have to specify a metaclass *explicitly* to get meta
+               behaviour. Example:
+
+                       class Foo:
+                               __class__ = MyMetaClass
+
+               as apposed to
+
+                       MyMeta = MyMetaClass("MyMeta", (), {})
+
+                       class Foo(MyMeta): pass
+
+               as it is with Guido's hook.
+
+               Reasons for this new hook:
+                       - Guido's hook abuses inheritance syntax, making it
+                         impossible to inherit from metaclasses without
special
+                         trickery.
+                       - implementing Meta stuff seems cleaner. Or maybe it's
+                         just me...
+
+               At first I thought Guido's hook would not be compatible with
+               mine, but they work together beautifully: inheritance works
+               just like you would expect.
+       */
+       {
+               PyObject *callable = NULL;
+               callable = PyDict_GetItemString(methods, "__class__");
+               if (callable) {
+                       PyObject *args;
+                       PyObject *newclass = NULL;
+                       PyDict_DelItemString(methods, "__class__");
+                       args = Py_BuildValue(
+                               "(OOO)", name, bases, methods);
+                       if (args != NULL) {
+                               newclass = PyEval_CallObject(
+                                       callable, args);
+                               Py_DECREF(args);
+                       }
+                       return newclass;
+               } else {
+                       PyErr_Clear();
+               }
+       }
+       /*      __END__ of Just's Hook */
        n = PyTuple_Size(bases);
        for (i = 0; i < n; i++) {
                PyObject *base = PyTuple_GET_ITEM(bases, i);
                if (!PyClass_Check(base)) {
                        /* Call the base's *type*, if it is callable.




From geldridg@progsoc.uts.edu.au  Tue Dec  1 12:17:41 1998
From: geldridg@progsoc.uts.edu.au (Geoff Eldridge)
Date: Tue, 1 Dec 1998 23:17:41 +1100 (EST)
Subject: [Types-sig] Re: [Types-sig]: HOWTO: viewing source posts on eGroups ..
In-Reply-To: <l03102802b2897d984c09@[193.78.237.139]>
Message-ID: <Pine.LNX.3.96.981201230424.11395B-100000@ftoomsh.progsoc.uts.edu.au>

On Tue, 1 Dec 1998, Just van Rossum wrote:

> At 11:12 PM -0500 11/30/98, Barry A. Warsaw wrote:
> >I haven't seen any comments on my posted syntax proposal; perhaps it's
> >gross, but anyway I liked it.  Or maybe because it's buried in a long
> >egroups posting:
> >
> >    <http://www.egroups.com/list/python-classes/54.html>
> 
> (Hm, since your post contains mostly code, and egroups screws it up
> majorly, I cannot read it... If you still have it, I'd sure appreciate it
> as prvt mail.)

Since this has come up a few time, you can view the source (link available
in top-right corner) of posts on eGroups by looking at the source (in this
case)..

   http://www.egroups.com/list/python-classes/54.html?raw=1

Just append the paramter ``?raw=1'' to the ``screwed up'' URL .. at least
that's how it works today (and since they went from findmail to eGroups
a few months back..) 

Geoff Eldridge

-- geldridg@progsoc.uts.edu.au
-- http://www.elj.com/new/



From Donald Beaudry <donb@tensilica.com>  Tue Dec  1 14:59:52 1998
From: Donald Beaudry <donb@tensilica.com> (Donald Beaudry)
Date: Tue, 01 Dec 1998 09:59:52 -0500
Subject: [Types-sig] Type/class unification
References: <000401be1be9$998df5a0$fa9e2299@tim>
Message-ID: <199812011459.JAA08934@hq.tensilica.com>

"Tim Peters" <tim_one@email.msn.com> wrote,
> 
> OTOH, from the Ruby reference manual (Class.html):
> 
>     To tell the truth, each class have the unnamed class (which
>     is the meta-class of the class), the class Class is the class
>     of these meta-classes. It is complicated, but it is not
>     important for using Ruby.
> 
> *That's* sure encouraging <wink>.

Hey, that sounds like something I might have written... two
years ago!

> wake-me-up-in-two-years-ly y'rs  - tim

Tim... Tim... it's time to wake up and smell the meta-classes ;)

	--Don



From guido@CNRI.Reston.VA.US  Tue Dec  1 15:18:37 1998
From: guido@CNRI.Reston.VA.US (Guido van Rossum)
Date: Tue, 01 Dec 1998 10:18:37 -0500
Subject: [Types-sig] Re: Meta-classes discussion starter
In-Reply-To: Your message of "Tue, 01 Dec 1998 12:16:04 +0100."
 <l03102802b2897d984c09@[193.78.237.139]>
References: <208801be1c9c$bbdfe100$2d232cd1@linwin.token.hapenney.com> <l03102801b288bbd9020d@[193.78.237.175]>
 <l03102802b2897d984c09@[193.78.237.139]>
Message-ID: <199812011518.KAA07076@eric.cnri.reston.va.us>

Interesting patch.  "Brotherly hooks" -- I like that :-)

Question for the types-sig readers: is it warranted to add this to the
next release?  I *could* smuggle it into 1.5.2 but I'm noty convinced.
It might even break existing code (Don B's tricks, or Jim F's
extension classes hooks).  Opinions?  Personally, I think I won't be
using or documenting this and I won't guarantee that it's still there
in 1.6 -- since in 2.0 things will probably be different (or the same, 
but that's hard to say -- the metaclasses stuff is 100% sure to change 
in 2.0 anyway and I won't promise b/w compatibility there).

--Guido van Rossum (home page: http://www.python.org/~guido/)


From Donald Beaudry <donb@tensilica.com>  Tue Dec  1 15:19:01 1998
From: Donald Beaudry <donb@tensilica.com> (Donald Beaudry)
Date: Tue, 01 Dec 1998 10:19:01 -0500
Subject: [Types-sig] Re: Meta-classes discussion starter
References: <209201be1c9e$dd6755e0$2d232cd1@linwin.token.hapenney.com>
Message-ID: <199812011519.KAA09191@hq.tensilica.com>

"Evan Simpson" <evan@tokenexchange.com> wrote,
> I can see how defining them, and specifiying their meta-classes, would solve
> the binding and naming problems.  You could even have multiple meta-classes
> just as you have multiple inheritance.

I guess you could, but since nobody can quite figure out why single
inheritance meta-classes are needed or what they might be used for,
I doubt that it would be a good idea!

> All you need now is a way to spell static methods.

How is specifying a static method any different than specifying a
method in a meta-class (for the benefit of the class)?  Or is that
what you are asking for?

A while back I proposed that meta-classes be specified by embedding
them in the class that needs it.  For example:

    class foo:
        class __class__:
            def some_class_method(self):
                # 'self' is a foo class object or one derived from it.
                ...

        def some_instance_method(self):
            # self
            ...

Then, foo.some_class_method() would be very similar to a 'static'
function in C++, while foo().some_instance_method() is just what we
expect today.  If a class was to be derived from foo, it would
"inherit" foo's meta-class.  More accurately though, any class derived
from foo would be an instance of the same meta-class as foo (unless of
course the derived class needs to specialize the meta-class).

(sorry if this has been discussed and tossed out already... I'm
 just starting to wake up)

	--Don


From Donald Beaudry <donb@tensilica.com>  Tue Dec  1 15:27:44 1998
From: Donald Beaudry <donb@tensilica.com> (Donald Beaudry)
Date: Tue, 01 Dec 1998 10:27:44 -0500
Subject: [Types-sig] Re: Meta-classes discussion starter
References: <l03102801b288bbd9020d@[193.78.237.175]>
Message-ID: <199812011527.KAA09346@hq.tensilica.com>

Just van Rossum <just@letterror.com> wrote,
> While digging in ceval.c and understanding how the Don Beaudry and
> Guido's own hook actually work I came up with mine, which I define thusly:
> 
> 	If the __dict__ of the class that is being built has a __class__
> 	key, call that thing with (name, bases, methods)
> 
> Which means you specify a metaclass like this:
> 
> class MyClass:
> 	__class__ = MyMetaClass
> 
> Not exactly pretty, but at least it doesn't look like inheritance. It also
> means that if you want to subclass from MyMetaClass you write

Fantastic!  Just, you're a genius!  ...and coming from a raving
lunatic, that might mean something, like you too are a raving lunatic.
That's exactly what I proposed a while back, but with a twist.  You
might be interested in talking a look at my objectmodule extension.

The twist is what you just proposed is allowed, you can then nest your
meta-class definition inside the class that needs it.  I just posted
an example of that in my last message to this sig.

	--Don


From Donald Beaudry <donb@tensilica.com>  Tue Dec  1 15:36:40 1998
From: Donald Beaudry <donb@tensilica.com> (Donald Beaudry)
Date: Tue, 01 Dec 1998 10:36:40 -0500
Subject: [Types-sig] Re: Meta-classes discussion starter
References: <208801be1c9c$bbdfe100$2d232cd1@linwin.token.hapenney.com> <l03102801b288bbd9020d@[193.78.237.175]> <l03102802b2897d984c09@[193.78.237.139]>
Message-ID: <199812011536.KAA09486@hq.tensilica.com>

Just van Rossum <just@letterror.com> wrote,
> >    JvR> (If anyone wants
> >    JvR> to play, I could post a patch.)
> >
> >Yes, please do!
> 
> See below... It's a against the current state of Python 1.5.2b1. It should
> be 100% compatible with the existing hooks (ie. I tried carefully not to
> break things).

It's interesting to note that a patch is not necessary.  All you need is
an extension module that exports a hand written meta-class.  This
meta-class can then poke into the class that's being defined and check
for the existence of the __class__ attribute.

been-there-done-that-ly y'rs,

	--Don



From gmcm@hypernet.com  Tue Dec  1 15:48:47 1998
From: gmcm@hypernet.com (Gordon McMillan)
Date: Tue, 1 Dec 1998 10:48:47 -0500
Subject: [Types-sig] Re: Types-SIG digest, Vol 1 #13 - 2 msgs
In-Reply-To: <000401be1ce3$47729c80$9b9e2299@tim>
References: <366263F4.31FAD263@aethos.co.uk>
Message-ID: <1299612608-19388054@hypernet.com>

> [Dominic Binks]
> > ...
> > I would hazard a guess that may be as much as 90% of Python code
> > is regularly type following.

[Tim Peters]

> I'd bet that's on the high side of the truth.  From the
> Walker/Griswold paper I  posted a link to over the weekend,
> 
>     An empirical study using the type inference system described
>     here was conducted on a large number of Icon programs written by
>     many different authors for a wide variety of applications. The
>     results, which are conservative, show a range of type
>     consistency from about 60 to 100%, with an average of about 80%.
>     That is, on the average, the operands of about 80% of the
>     operators in these programs always have the same type.
> 
> It's likely lower in Python, because (I believe) Python programmers
> write more polymorphic functions than do Icon programmers.

From my own code, I'd guess that at least 90% of the time type(x) 
is constant. But where type(x) == InstanceType or ClassType, I'd 
guess that less that 50% of the time is x.__class__ constant, and 
frequently the values are not even related by inheritance.

So resolving the class / type problem means interfaces are necessary 
for optional static typing, while interfaces changes the obvious way 
to do optional static typing and probably also the reformation of the 
class model that goes with the type / class reformation, and optional 
static typing probably affects the other two similarly.

time-for-a-little-deus-ex-machina-ly y'rs

- Gordon


From tismer@appliedbiometrics.com  Tue Dec  1 16:24:20 1998
From: tismer@appliedbiometrics.com (Christian Tismer)
Date: Tue, 01 Dec 1998 17:24:20 +0100
Subject: [Types-sig] Re: Meta-classes discussion starter
References: <208801be1c9c$bbdfe100$2d232cd1@linwin.token.hapenney.com>
 <l03102801b288bbd9020d@[193.78.237.175]> <l03102802b2897d984c09@[193.78.237.139]>
Message-ID: <36641834.45EA8316@appliedbiometrics.com>

...
> >    JvR> Which means you specify a metaclass like this:
> >
> >    | class MyClass:
> >    |  __class__ = MyMetaClass
> >

Just, to me this looks absolutely perfect.
And I can understand its behavior immediately.
It makes metaclasses no longer be a brain-teaser.

I hope that many people think so as well - cheers - chris

-- 
Christian Tismer             :^)   <mailto:tismer@appliedbiometrics.com>
Applied Biometrics GmbH      :     Have a break! Take a ride on Python's
Kaiserin-Augusta-Allee 101   :    *Starship* http://starship.skyport.net
10553 Berlin                 :     PGP key -> http://pgp.ai.mit.edu/
     we're tired of banana software - shipped green, ripens at home


From just@letterror.com  Tue Dec  1 17:39:40 1998
From: just@letterror.com (Just van Rossum)
Date: Tue, 1 Dec 1998 18:39:40 +0100
Subject: [Types-sig] Re: Meta-classes discussion starter
In-Reply-To: <199812011536.KAA09486@hq.tensilica.com>
References: <208801be1c9c$bbdfe100$2d232cd1@linwin.token.hapenney.com>
 <l03102801b288bbd9020d@[193.78.237.175]>
 <l03102802b2897d984c09@[193.78.237.139]>
Message-ID: <l03102803b289d6456f0b@[193.78.237.134]>

At 10:36 AM -0500 12/1/98, Donald Beaudry wrote:
>It's interesting to note that a patch is not necessary.  All you need is
>an extension module that exports a hand written meta-class.  This
>meta-class can then poke into the class that's being defined and check
>for the existence of the __class__ attribute.

But then you don't have the advantage which was the main reason to make
that patch, which is that you don't have to spell it like:

class A(some_extension_object):
    __class__ = whatever

but that the following "just works":

class A:
    __class__ = mymetahook

Or do I miss something here?

On a related note, would my hook break your stuff? The test to see whether
the method dict contains a __class__ entry happens *before* the tests that
check whether one of the base classes either has a callable type or has a
__class__ attr. If it does (break your stuff), maybe I should reverse the
tests, although that would make the hook less elegant.

Just




From da@skivs.ski.org  Tue Dec  1 17:55:40 1998
From: da@skivs.ski.org (David Ascher)
Date: Tue, 1 Dec 1998 09:55:40 -0800 (Pacific Standard Time)
Subject: [Types-sig] Re: Meta-classes discussion starter
In-Reply-To: <l03102802b2897d984c09@[193.78.237.139]>
Message-ID: <Pine.WNT.4.04.9812010953340.211-100000@rigoletto.ski.org>

On Tue, 1 Dec 1998, Just van Rossum wrote:

> At 11:12 PM -0500 11/30/98, Barry A. Warsaw wrote:
>
> >    <http://www.egroups.com/list/python-classes/54.html>
> 
> (Hm, since your post contains mostly code, and egroups screws it up
> majorly, I cannot read it... If you still have it, I'd sure appreciate it
> as prvt mail.)

FYI: in egroups, if you choose the 'source' link, you get the raw ASCII,
w/ indentation and all that.

Also FYI, the archives of the egroups list will show up on python.org in a
reasonable format someday.

Also FYI: if you want to get specific (or all) archives from the egroups
list, use:

import urllib, string

for x in range(1, 87):
    print "getting post %d" % x
    data = urllib.urlopen('http://www.egroups.com/list/python-classes/%d.html?raw=1' % x).read()
    while 1:
	try:
	    core = data[string.index(data, '<pre>')+5 : string.index(data, '</pre>')-1]
	    core = string.join(string.split(core, '\n')[:-5], '\n')
	    open('%d.msg' % x, 'w').write(core)
	    break;
	except:
	    print "eGroups wants validation -- ignore it"
	    data = urllib.urlopen('http://www.egroups.com/list/python-classes/%d.html?raw=1' % x).read()




From jim@Digicool.com  Tue Dec  1 18:36:09 1998
From: jim@Digicool.com (Jim Fulton)
Date: Tue, 01 Dec 1998 18:36:09 +0000
Subject: [Types-sig] Interfaces: Scarecrow implementation v 0.1 is available
Message-ID: <36643719.E24A7CD3@digicool.com>

Version 0.1 of my implementation of the Scarecrow proposal is now available at:

  http://www.digicool.com/releases/Interface-0.1.tar.gz

and at:

  ftp://ftp.digicool.com/pub/releases/Interface-0.1.tar.gz

Here are the README.txt contents.


Python Interfaces - The Scarecrow Implementation

This document describes my implementation of the Python interfaces
scarecrow proposal.  

Status

  This is a first-cut implementation of the proposal.  My primary goal
  is to shed light on some ideas and to provide a framework for
  concrete discussion.

  This implementation has had minimal testing. I expect many aspects
  of the implementation to evolve over time.

Basic assumptions:

  Interfaces are *not* classes:
    
    o Interfaces have their own "hierarchy" (DAG really)
    
    o Interfaces are objects that provide a protocol for
    	querying attributes (including methods) defined by an
    	an interface:
    
    	  names() -- return a sequence of defined names
    
    	  getDescriptionFor(name, [default]) --
    	     Get a description of a name.
    
    o You cannot mix interfaces and classes in base-class lists.
    
  There are utilities and methods for computing implied interfaces
  from classes and for computing "defered" classes from interfaces.

  Why aren't interface classes?  Interfaces perform a different
  function that classes.  Classes are for sharing implementation.
  Interfaces are for denoting, defining, and documenting abstract
  behavior.      

Details

  Software layout

    There is an 'Interface' package that exports a variety of useful
    facilities.  These are described below.

  Creating Interfaces

    Interfaces can be created in several ways.  The class statement
    can be used with one or more interfaces provided as base classes.
    This approach is convenient, syntactically, although it is a
    little missleading, since interfaces are *not* classes.  A minimal
    interface that can be used as a base is Interface.Base.
  
    You can also call Interface.new:
  
      new(name, [bases, attrs, __doc__]) --
  
	Create a new interface.  The arguments are:
  
	  name -- the interface name
  
	  bases -- a sequence of "base" interfaces.  Base interfaces
	    are "extended" by the new interface.
  
	  attrs -- an object that conforms to
	    'Interfaces.Standard.Dictionary' that provides attributes
	    defined by an interface.  The attributes should be
	    'Interface.Attribute objects'.
  
    Finally you can compute an implied interface from a class by calling
    'Interface.impliedInterface': 
  
      impliedInterface(klass, [__name__, __doc__])
  
	 klass -- a class from which to create an interface.
  
	 __name__ -- The name of the interface.  The default name is the
	    class name with the suffix "Interface".
  
	__doc__ -- a doc string for the interface.  The default doc
	    string is the class doc string.
  
	The generated interface has attributes for each public method
	defined in or inherited by the interface. A method is considered
	public if it has a non-empty doc string and if it's name does
	not begin with '_' or does begin and end with '__' and is
	greater than 4 characters in length.
  
    Note that computing an interface from a class does not automatically
    assert that the class implements an interface.

    Here's an example:

      class X:

        def foo(self, a, b):
          ...

      XInterface=Interface.impliedInterface(X)
      X.__implements__=XInterface

  Interface assertions

    Objects can assert that they implement one or more interfaces.
    They do this by by defining an '__interfaces__' attribute that is
    bound to an interface assertion.

    An *interface assertion* is either: 

      - an Interface or

      - a sequence of interface assertions.

    Here are some examples of interface assertions:

      I1

      I1, I2

      I1, (I2, I3)

    where I1, I2, and I3 are interfaces.

    Classes may provide (default) assertions for their instances
    (and subclass instances).  The usual inheritence rules apply.
    Note that the definition of interface assertions makes composition
    of interfaces straightforword.  For example:

      class A:

        __implements__ = I1, I2 

        ...

      class B

        __implements__ = I3, I4

      class C(A. B):
        ...

      class D:
        
        __implements__ = I5

      class E:

        __implements__ = I5, A.__implements__
      
  Special-case handling of classes

    Special handling is required for Python classes to make assertions
    about the interfaces a class implements, as opposed to the
    interfaces that the instances of the class implement.  You cannot
    simply define an '__implements__' attribute for the class because
    class "attributes" apply to instances.

    By default, classes are assumed to implement the Interface.Standard.Class
    interface.  A class may override the default by providing a
    '__class_implements__' attribute which will be treated as if it were
    the '__implements__' attribute of the class.

  Testing assertions

    You can test whether an object implements an interface by calling
    the 'implementedBy' method of the interface and passing the
    object::

      I1.implementedBy(x)

    Similarly, you can test whether, by default, instances of a class
    implement an interface by calling the 'implementedByInstancesOf'
    method on the interface and passing the class::
  
      I1.implementedByInstancesOf(A)

  Testing interfaces

    You can test whether one interface extends another by calling the
    extends method on an interface:

      I1.extends(I2)

    Note that an interface does not extend itself.

  Interface attributes

    The purpose of an interface is to describe behavior, not to
    provide implementation.  In a similar fashion the attributes of
    an interface describe and document the attributes provided by an
    object that implements the interface.

    There are currently two kinds of supported attributes:

      Interface.Attribute -- The objects describe interface
        attributes.  They define at least names and doc strings and
        may define other information as well.

      Interface.Method -- These are interface attributes that
        describe methods.  They *may* define information about method
        signatures. (Note Methods are kinds of Attributes.)

    When a class statement is used to define an interface, method
    definitions may be provided.  These get converted to Method
    objects during interface creation.  For examle:

      class I1(Interface.Base):
         
        __name__=Attribute("The object's name")

        def foo(self, a, b):
           "blah blah"

    defines an interface, 'I1' that has two attributes, '__name__' and
    'foo'. The attribute 'foo' is a Method instance.  It is *not* a
    Python method.

    It is my expectation that Attribute objects will eventually be
    able to provide all sorts of interesting meta-data.  

  Defered classes

    You cannot use interfaces as base classes.  You can, however, 
    create "defered" classes from an interface:

      class StackInterface(Interface.Base):

         def push(self, v):
            "Add a value to the top of a stack"

         def pop(self):
            "Remove and return an object from the top of the stack"

      class Stack(StackInterface.defered()):
         "This is supposed to implement a stack"

         __implements__=StackInterface

    Attempts to call methods inherited from a defered class will
    raise Interface.BrokenImplementation exceptions.

  Trial baloon: abstract implementations

    Tim Peter's has expressed the desire to provide abstract
    implementations in an interface definitions, where, presumably, an
    abstract implementation uses only features defined by the
    interface.  For example:

      class ListInterface(Interface.Standard.MutableSequence):

        def append(self, v):
           "add a value to the end of the object"

	def push(self, v):
           "add a value to the end of the object"
           self.append(v)

    Perhaps if a method definition has a body (other than a doc
    string) then the corresponding method in the defered class
    will not be defered. This would not be hard to do in CPython
    if I cheat and sniff at method bytecodes.

  Standard interfaces

    The module Interface.Standard defines interfaces for standard
    python obnjects.

    This module, and the modules it uses need a lot more work!

  Handling existing built-in types

    A hack is provided to allow implementation assertions to be made
    for builtin types.  Interfaces.assertTypeImplements can be called
    to assert that instances of a built-in type implement one or more
    interfaces::

       Util.assertTypeImplements(
         type(1L), 
         (AbritraryPrecision, BitNumber, Signed))


    

Issues

  o What should the objects that define attributes look like?
    They shouldn't *be* the attributes, but should describe the
    the attributes.

    Note that we've made a first cut with 'Attribute' and
    'Method' objects.

  o There are places in the current implementation that use
    'isinstance' that should be changed to use interface
    checks.

  o When the interface interfaces are finalized, C implementations
    will be highly desirable for performance reasons.

  o Alot more work is needed on the standard interface hierarchy.    

  ...



Jim

--
Jim Fulton           mailto:jim@digicool.com
Technical Director   (540) 371-6909              Python Powered!
Digital Creations    http://www.digicool.com     http://www.python.org

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission.  Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.


From Donald Beaudry <donb@tensilica.com>  Tue Dec  1 18:49:00 1998
From: Donald Beaudry <donb@tensilica.com> (Donald Beaudry)
Date: Tue, 01 Dec 1998 13:49:00 -0500
Subject: [Types-sig] Re: Meta-classes discussion starter
References: <208801be1c9c$bbdfe100$2d232cd1@linwin.token.hapenney.com> <l03102801b288bbd9020d@[193.78.237.175]> <l03102802b2897d984c09@[193.78.237.139]> <l03102803b289d6456f0b@[193.78.237.134]>
Message-ID: <199812011849.NAA12362@hq.tensilica.com>

Just van Rossum <just@letterror.com> wrote,
> At 10:36 AM -0500 12/1/98, Donald Beaudry wrote:
> >It's interesting to note that a patch is not necessary.  All you need is
> >an extension module that exports a hand written meta-class.  This
> >meta-class can then poke into the class that's being defined and check
> >for the existence of the __class__ attribute.
> 
> But then you don't have the advantage which was the main reason to make
> that patch, which is that you don't have to spell it like:
> 
> class A(some_extension_object):
>     __class__ = whatever
> 
> but that the following "just works":
> 
> class A:
>     __class__ = mymetahook
> 
> Or do I miss something here?

Nope... I dont think you are missing anything.  I just dont mind using
some_extension_object as base class as much as you do.  

> On a related note, would my hook break your stuff?

I guess that depends on how you define "break" ;) The stuff I did was
mostly an experiment to play with the syntax for defining
meta-classes.  With your change it's likely that it wont work as is,
but I doubt that it works with 1.5 anyways (though I might have fixed
it, I dont really remember).  For the most part, it has served its
purpose.  If more people look at it, it might continue to serve its
purpose... it does, afterall, raise a few more issues.

> The test to see whether the method dict contains a __class__ entry
> happens *before* the tests that check whether one of the base
> classes either has a callable type or has a __class__ attr. If it
> does (break your stuff), maybe I should reverse the tests, although
> that would make the hook less elegant.

I suppose we could put off this part of the discussion until Guido
says that he's interested in the patch.

	--Don


From just@letterror.com  Tue Dec  1 19:51:05 1998
From: just@letterror.com (Just van Rossum)
Date: Tue, 1 Dec 1998 20:51:05 +0100
Subject: [Types-sig] Re: Meta-classes discussion starter
In-Reply-To: <199812011518.KAA07076@eric.cnri.reston.va.us>
References: Your message of "Tue, 01 Dec 1998 12:16:04 +0100."
 <l03102802b2897d984c09@[193.78.237.139]>
 <208801be1c9c$bbdfe100$2d232cd1@linwin.token.hapenney.com>
 <l03102801b288bbd9020d@[193.78.237.175]>
 <l03102802b2897d984c09@[193.78.237.139]>
Message-ID: <l03102809b289e8cec819@[193.78.237.134]>

At 10:18 AM -0500 12/1/98, Guido van Rossum wrote:
>Question for the types-sig readers: is it warranted to add this to the
>next release?  I *could* smuggle it into 1.5.2 but I'm noty convinced.
>It might even break existing code (Don B's tricks, or Jim F's
>extension classes hooks).  Opinions?

Maybe it breaks Donald's stuff, since he also uses __class__ to mean
things. If it does break, I'd be happy to call it something else (maybe
"__klass__").

>Personally, I think I won't be
>using or documenting this and I won't guarantee that it's still there
>in 1.6 -- since in 2.0 things will probably be different (or the same,
>but that's hard to say -- the metaclasses stuff is 100% sure to change
>in 2.0 anyway and I won't promise b/w compatibility there).

It's a hack/hook for now, which should be replaced with real syntax in 2.0
unless _all_ proposals are uglier than using __class__ ;-). B/w
compatibility is not an issue. My main reason for having it now is to be
able to "easily" experiment with metaclasses, maybe it's even possible to
prototype an object model for Python 2.0 using Python 1.x. Right now I'm
trying to implement the current class protocol _using_metaclasses_, and
once that works I'd like to imitate the current instance protocol, again
using metaclasses. My head is still exploding at regular intervals, but it
is a great deal more complicated than Meta.py, which really only implements
the instance protocol,  using regular classes... Now, if I get *that* to
actually work, I'll write something up and write some demo's.

Just




From jim@Digicool.com  Tue Dec  1 21:03:39 1998
From: jim@Digicool.com (Jim Fulton)
Date: Tue, 01 Dec 1998 21:03:39 +0000
Subject: [Types-sig] RE: PRE-PROPOSAL: Python Interfaces
References: <000201be1663$827f3260$2f9e2299@tim>
Message-ID: <366459AB.BD35A34@digicool.com>

Tim Peters wrote:
> 

(snip)

> OTOH, unlike e.g. JimF I have nothing against stuffing some default
> implementation into interfaces.  To the contrary, if I *wanted* to say that
> all List implementers implement Stack too, I think
> 
>    class List(Interface, Stack):  #??? spelling of interface

I assume that List, Stack and Interface are all interfaces.
BTW, I ended up spelling the base interface 'Interface.Base'.

>       __implements__ = Stack      #??? ... hierarchy is unclear

What does this want to mean?

>       def pop(self): raiseDeferredError()
>       def append(self, thing): raiseDeferredError()
>         ...
>       def push(self, thing): self.append(thing)
> 
> is the obvious way to do it.  This doesn't preclude a different
> implementation of push, it simply reflects the truth that List *can* give
> you an implementation of push guaranteed to meet push's spec, assuming only
> that the implementations of List.append and List.pop meet *their* specs.
> 
> For a simpler example, after rich comparisons are in
> 
>     class SaneOrdering(Interface):
>         def __eq__(self, other): raiseDeferredError()
>         def __ne__(self, other): return not self.__eq__(other)
> 
> would save 99% of the SaneOrderings ever created the pointless and
> error-prone bother of implementing __ne__ themselves.  That is, if the
> post-condition for SaneOrdering.__ne__(self, other) is
> 
>     assert Result == not self.__eq__(other)
> 
> what better way to encourage correct software than to provide a default
> implementation that guarantees it without further ado?  Many methods in rich
> interfaces exist simply for user convenience, having obvious (but clumsy to
> type out) implementations in terms of other methods.

I've tried to provide a compromise in the implementation I released 
earlier today.

I provide a method, 'defered' on interfaces that returns a defered classes.
For example:

  class StackInterface(Interface.Base):
     
    def pop(self):
       "Remove and return"

    def append(self, v):
       "Add an object to the stack"

    def push(self, v):
       "Add an object to the stack"
 
  class Stack(StackInterface.defered()):

    def push(self, v): ...
    def pop(self):  ...

  s=Stack()
  s.append(1) 

raises a BrokenImplementation error rather than
an AttributeError because the author of the Stack 
class failed to provide an append method.

I've also suggested that if a method definition in
an interface contains a body (other than a doc string), 
then perhaps a non-defered method will be provided when 
a defered class is genrated from the interface.  Note 
that the body should only use attributes defined by the 
interface.  For example, if the append method 
defined in StackInterface above was:

    def append(self, v):
       "Add an object to the stack"
       self.push(v)

then the defered class would have a non-defered implementation
of append and:

  a.append(1)

would suceed.

Jim

--
Jim Fulton           mailto:jim@digicool.com
Technical Director   (540) 371-6909              Python Powered!
Digital Creations    http://www.digicool.com     http://www.python.org

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission.  Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.


From jim@Digicool.com  Tue Dec  1 21:13:25 1998
From: jim@Digicool.com (Jim Fulton)
Date: Tue, 01 Dec 1998 21:13:25 +0000
Subject: [Types-sig] Re: PRE-PROPOSAL: Python Interfaces
References: <13915.20041.23787.219850@anthem.cnri.reston.va.us>
 <000501be18c6$a0bbb540$cb9e2299@tim> <13923.26171.828432.697396@anthem.cnri.reston.va.us>
Message-ID: <36645BF5.37816AD2@digicool.com>

Barry A. Warsaw wrote:
> 
> >>>>> "TP" == Tim Peters <tim_one@email.msn.com> writes:
> 
>     TP> 7/16'ths of me too, but I believe the groundrules for the
>     TP> Scarecrow Proposal include "no language changes".  That leaves
>     TP> manual, or at best semi-automated, convention for now.  In a
>     TP> Python2 world, I doubt it would be more of an insanely
>     TP> protracted battle than usual <wink> to get a "defer" keyword
>     TP> added, provided it worked out in practice by hand first.
> 
>     TP> Given "def push(self, thing): defer" we could peek at the
>     TP> bytecode today, note the senseless LOAD_GLOBAL of "defer", and
>     TP> simply rewrite the user's code for them <cackle>.
> 
> An easy change for Python 1.6 would be to add something like
> NotImplementedError to exceptions.py and a defer() builtin function.
> The latter has some potential for existing code breakage, but since
> user definitions of that attribute will silently override the builtin
> one, it's probably minimal.  That might get us far enough along for
> 1.6 to play while leaving open a `defer' keyword later.
> 
> So trivial to implement, I've attached a patch set (against my hacked
> 1.5.2 -- watch for fuzz factor).  Extremely minimally tested.  Here's
> what it looks like:

The Scarecrow implementation I posted earlier today provides this
functionality without any language or library changes.

Jim

--
Jim Fulton           mailto:jim@digicool.com
Technical Director   (540) 371-6909              Python Powered!
Digital Creations    http://www.digicool.com     http://www.python.org

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission.  Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.


From jim@Digicool.com  Tue Dec  1 21:19:50 1998
From: jim@Digicool.com (Jim Fulton)
Date: Tue, 01 Dec 1998 21:19:50 +0000
Subject: [Types-sig] Scarecrow Proposal
References: <3657ECA1.6B2BD52@lemburg.com>
 <365A0634.9D7B079F@digicool.com>
 <365A7E58.7111F703@lemburg.com>
 <365AAE8D.6D38911B@digicool.com>
 <13915.14710.82888.768798@anthem.cnri.reston.va.us>
 <365B43B2.EAFE1331@digicool.com> <13915.20257.865977.315618@anthem.cnri.reston.va.us>
Message-ID: <36645D76.3ACEA8C3@digicool.com>

Barry A. Warsaw wrote:
> 
> >>>>> "JF" == Jim Fulton <jim@Digicool.com> writes:
> 
>     >> Barry A. Warsaw wrote:
>     >> Doesn't it make sense for introspection purposes to have access
>     >> to the methods defined in an interface through the interface
>     >> object?
> 
>     JF> Of course, but *not* as attributes of the interface.  The
>     JF> interface will provide methods for getting these.
> 
> Okay, but then the objects those methods return will have to have all
> the (appropriate) attributes one might expect to be able to get from a
> "normal" method.

They will have an interface for getting all that information.
Exactly what that interface should be is open for debate.
It probably shouldn't involve code objects or other artifacts
of the current CPython implementation.  At one time, Guido
seemed not to like people to use things like func_code, func_defaults, 
etc.

Jim

--
Jim Fulton           mailto:jim@digicool.com
Technical Director   (540) 371-6909              Python Powered!
Digital Creations    http://www.digicool.com     http://www.python.org

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission.  Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.


From mengx@dun.nielsen.com  Tue Dec  1 21:44:56 1998
From: mengx@dun.nielsen.com (Ted Meng)
Date: Tue, 1 Dec 1998 16:44:56 -0500
Subject: [Types-sig] optional typing and performance
Message-ID: <199812012144.QAA18669@p5mts.nielsenmedia.com>

The actual format for type declaration does not matter to me
 as long as parser/compiler can recognize it. 
But I'd rather see the types declared in the doc string
since it would not change Python grammar which eases compatibility(so that
a module written for 2.0 with types could be usable for 1.5.1).

It also make experiments(adding/droping) for type proposals a lot of easier.
Since the type declarations are embedded in the doc string, the type
declarations might be added documented dynamically in FAQ 
instead of Reference manual
as performance tuning tips to efficiently use Python interpreter/compiler.

The whole point is: Python is simple, practical, yet powerful, let's
keep it that way otherwise we better switch to Java now.

 


> From poplib  Mon Nov 30 11:35:04 1998
> Date: Mon, 30 Nov 1998 17:35:11 +0100
> From: Martijn Faassen <faassen@vet.uu.nl>
> X-Accept-Language: en
> MIME-Version: 1.0
> To: Xiangyang Ted Meng <mengx@nielsenmedia.com>
> Subject: Re: [Types-sig] optional typing and performance
> Content-Transfer-Encoding: 7bit
> X-UIDL: 822534a483a263f3c01eaa26d665c685
> 
> Xiangyang Ted Meng wrote:
> > 
> > Hi,
> > 
> > I am curious that if it was brought up before and what loop holes
> > it might have. Is it possible to embed optional type declarations
> > in the doc string, something like:
> > 
> > def function(a,b,c):
> >         '''
> >         (normal documentations)
> > 
> >         [start of python optional  types]
> > 
> >         function: int
> >         a: int, b: String, c: int
> >         d: int
> > 
> >         [end of python optional  types]
> > 
> >         '''
> > 
> >         d=a+c
> > 
> >         return d
> > 
> > It might not look good for a language designer.
> > But compared to the proposed ADD-LATER typing ( I only took
> > glimpse of the WEB posting, might have missed something)
> > like below
> > 
> > def function: int (a:int,b:String,c:int):
> > 
> >         '''
> >         (normal documentations)
> >         '''
> >         d:int =a+c
> > 
> >         return d
> 
> I agree the optional static typing proposals I've seen for Python so far
> look scary; it doesn't look much like Python, as it clutters up stuff a
> lot. Perhaps it's just that I'm not used to them yet, but I fear it
> isn't.
> 
> An alternative to using doc strings is to use a kind of old style K&R C
> style typing (was this optional too in the beginning? I suppose so!),
> mixed with some Pascallish, and docstrings:
> 
>     def function(a, b):
>         var:
>             Result: int   "The sum of a and b."  
>             a, b: int     "The things that need to be added."
>             sum: int      "Local variable used to temporarily store the
> sum."
>         """This function uses optional static type checking.
>         """
> 
>         sum = a + b
>         return sum        
>                 
> The 'var' block can safely be ignored and the program should still work.
> IDEs can even 'fold' it away, and preprocessors can rip all var blocks
> out. Docstrings can be used to document what variables are used for, so
> even this would be possible, going beyond just static typing into
> documentation:
> 
>     def function():
>         var:
>             Result "Absolutely anything can be returned"
>  
> I've used 'Result' instead of the function name, for easy copy & pasting
> of 'var' blocks. I know one shouldn't copy & paste so much it to prevent
> 'rape & paste programming', but everybody does it at times, right? var
> blocks can be copied & pasted more easily than intermixed static type
> annotations. To abstract interface, one could split the 'var' block into
> an 'signature' (or 'arg') and 'local' block, like this:
> 
>     def function(a, b):
>        signature:
>            a, b: int   "Stuff to process"
>            Result: int "Result of the entire calculation."
>        local:
>            foo, bar, baz: int "Lots of helper variables"
>        """This function does a whole lot of silly stuff to a and b.
>        """
>        
>        foo = a + b * a
>        bar = foo * 12
>        baz = foo + bar
>        return baz
> 
> Another function could share the same signature.
> 
> Python could even figure out which functions are 'fully statically
> typed', and pass them on to the Python compiler. These functions can
> still easily be prototyped in Python, and the 'signature' and 'local'
> blocks would only need to be added when the function stabilizes and we
> need the speed. A tool could automate this process, by pointing out to
> the developer which variables in a function haven't been described yet:
>    hocheck test.py
>    
>    Line 173, cannot hyperoptimize function foo; bar has no static type.
>    Line 214, cannot hyperoptimize function xyz; i, j have no static
> type.
> 
> Excercises for the reader:
> 
> * Extend this to classes
> * Make Python interfaces fit in with this scheme
> 
> Is-this-more-in-the-spirit-of-Python?-ly yours,
> 
> Martijn
> 


From just@letterror.com  Tue Dec  1 21:57:18 1998
From: just@letterror.com (Just van Rossum)
Date: Tue, 1 Dec 1998 22:57:18 +0100
Subject: [Types-sig] MetaClasses (Non-ASCII art)
Message-ID: <l0310280cb28a106c18a2@[193.78.237.134]>

I made a drawing of a possible meta object model (...or what I understand
of it), but it went beyond ASCII, so you are kindly invited to visit:
	http://starship.skyport.net/~just/meta/

Whether the dotted lines actually actually go through the objects or not is
left vague on purpous. It depends on whether you'd like to write
	instance.__class__
	instance.__turtle__
or
	instance->in_class
	instance->ob_type
or
	instance.__class__
	instance.__class__.__nature__
or (my fave, so this matches the drawing best)
	instance.__class__
	instance.__class__.__class__

Inheritance is deliberately left out of the picture, since it is defined
(according to my model) by the metametaclass.

Does any of this make any sense to anyone?

Just




From skaller@maxtal.com.au  Wed Dec  2 08:17:55 1998
From: skaller@maxtal.com.au (John Skaller)
Date: Wed, 02 Dec 1998 18:17:55 +1000
Subject: [Types-sig] Re: Meta-classes discussion starter
Message-ID: <1.5.4.32.19981202081755.00f58f34@triode.net.au>

At 05:10 1/12/98 -0500, Tim Peters wrote:
>[John Skaller, deconstructs classes and finds ... no, I won't
> spoil the surprise <wink>]
>
>> ...
>> And that is it. :-) There is no such thing as a class :-))
>> It is nothing more than a function which makes objects.
>> ...
>
>John, Python is not a functional language, and there's not a shadow of a
>hint that it will ever become one.

        You're not following. I'm analysing the _structure_ of a system.
I'm telling you classes are _isomorphic_ to functions; and providing
the isomorphism. I didn't claim 'python was a functional language'.

>I would welcome a few more carefully chosen functional features myself, but
>"everything's a function" is so obviously not in Python's worldview 

        Excuse me. Python is a piece of software. It doesn't have a worldview.
Perhaps _you_ do. If it isn't in your worldview, fine.

>that
>this crusade has about as much chance of success as a "and how do we express
>functions? 

        This is not a 'crusade'. It's an analysis.

>> If we had these kinds of constructions in python, we could build
>> functions which generate functions more easily than we can today
>> [using exec, or by emulating these things with classes]. And then
>> we would have a better meta-programming system.
>
>What we would have is another functional language, a niche already fatally
>over-served by others.  They're elegant!  Succinct!  Powerful!  Gorgeous!
>Virtually unused!  Most people find them hostile and austere.  The repeated
>failure of functional languages to catch on over 25+ glorious years is hard
>to wish away.  You would like them, though!  I sure did (but I don't often
>confuse them with Python, and when I do Guido ignores me until I snap out of
>it ...).

        Now who is ranting? <g>

        My suggestion is for more builtin functions, of the kind needed
to construct functions.

>>         Or, you can turn that around, and consider functions
>> as a special kind of object:
>
>That's been Python's worldview from the start.

        Sigh. If functions and objects are consider _dual_,
then it's clear you need a balanced set of features to support
both functions and objects.

        Python has reasonable support for objects. It could do with
more support for functions. Or would it be more 'appealing' if I called
them methods?

>> and my very point is that we need primitives to operate on this
>> kind of object and we don't have them.
>
>How about a specific proposal?  You've made the high-concept "series and
>parallel and switch" pitch before, but until there's some meat on its bones
>it will most likely get tossed to the dogs <woof>.

        There is some code available for browsing.
-------------------------------------------------------
John Skaller    email: skaller@maxtal.com.au
		http://www.maxtal.com.au/~skaller
		phone: 61-2-96600850
		snail: 10/1 Toxteth Rd, Glebe NSW 2037, Australia




From skaller@maxtal.com.au  Wed Dec  2 08:18:19 1998
From: skaller@maxtal.com.au (John Skaller)
Date: Wed, 02 Dec 1998 18:18:19 +1000
Subject: [Types-sig] Interfaces: Scarecrow implementation v 0.1 is
 available
Message-ID: <1.5.4.32.19981202081819.00f6ba90@triode.net.au>

[Scarecrow v 0.1]

I'll try to add this to interscript, and integrate it with my protocols
module. :-)
Sigh. It's a special case of a protocol.

>  Special-case handling of classes
>
>    Special handling is required for Python classes to make assertions
>    about the interfaces a class implements, as opposed to the
>    interfaces that the instances of the class implement.  You cannot
>    simply define an '__implements__' attribute for the class because
>    class "attributes" apply to instances.

        Yes you can. And you must. See below.

>    By default, classes are assumed to implement the Interface.Standard.Class
>    interface.  A class may override the default by providing a
>    '__class_implements__' attribute which will be treated as if it were
>    the '__implements__' attribute of the class.

        This cannot work. What you need to do is fix the lookup routines,
that is, the routines that test if an object provides an interface, etc,
so that they look in the dictionary of an object directly!

        Don't use 'getattr', use

        object.__dict__.has_key('__implements__')

and

        object.__class__.__dict__.has_key('__class_implements__')

        This works, it is what I do in my protocols module,
and it gets rid of the special case, which is a sure sign of a design fault.


>  Trial baloon: abstract implementations
>
>    Tim Peter's has expressed the desire to provide abstract
>    implementations in an interface definitions, where, presumably, an
>    abstract implementation uses only features defined by the
>    interface.  For example:
>
>      class ListInterface(Interface.Standard.MutableSequence):
>
>        def append(self, v):
>           "add a value to the end of the object"
>
>	def push(self, v):
>           "add a value to the end of the object"
>           self.append(v)

        Yes. This is useful. It is the basis for mixins in C++.
But one has to ask the question: why not just use a class,
and add a 'defer' keyword to Python.

        Then again, you could just say 'pass'. 

>Issues
>
>  o What should the objects that define attributes look like?
>    They shouldn't *be* the attributes, but should describe the
>    the attributes.

        Obviously, they should themselves be interfaces.
Since attributes are just objects. :-)
-------------------------------------------------------
John Skaller    email: skaller@maxtal.com.au
		http://www.maxtal.com.au/~skaller
		phone: 61-2-96600850
		snail: 10/1 Toxteth Rd, Glebe NSW 2037, Australia




From mal@lemburg.com  Wed Dec  2 09:31:28 1998
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 02 Dec 1998 10:31:28 +0100
Subject: [Types-sig] Re: Meta-classes discussion starter
References: <208801be1c9c$bbdfe100$2d232cd1@linwin.token.hapenney.com>
 <l03102801b288bbd9020d@[193.78.237.175]> <l03102802b2897d984c09@[193.78.237.139]>
Message-ID: <366508F0.4B896660@lemburg.com>

Just van Rossum wrote:
> 
> +       {
> +               PyObject *callable = NULL;
> +               callable = PyDict_GetItemString(methods, "__class__");
> +               if (callable) {
> +                       PyObject *args;
> +                       PyObject *newclass = NULL;
> +                       PyDict_DelItemString(methods, "__class__");
> +                       args = Py_BuildValue(
> +                               "(OOO)", name, bases, methods);
> +                       if (args != NULL) {
> +                               newclass = PyEval_CallObject(
> +                                       callable, args);
> +                               Py_DECREF(args);
> +                       }
> +                       return newclass;
> +               } else {
> +                       PyErr_Clear();
> +               }
> +       }
> +       /*      __END__ of Just's Hook */

I like the patch... but why do you delete the __class__ attribute ?
To prevent endless recursion in some weird cases ?

-- 
Marc-Andre Lemburg                               Y2000: 394 days left
---------------------------------------------------------------------
          : Python Pages >>> http://starship.skyport.net/~lemburg/  :
           ---------------------------------------------------------



From mal@lemburg.com  Wed Dec  2 09:40:04 1998
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 02 Dec 1998 10:40:04 +0100
Subject: [Types-sig] MetaClasses (Non-ASCII art)
References: <l0310280cb28a106c18a2@[193.78.237.134]>
Message-ID: <36650AF4.6C6A8519@lemburg.com>

Just van Rossum wrote:
> 
> I made a drawing of a possible meta object model (...or what I understand
> of it), but it went beyond ASCII, so you are kindly invited to visit:
>         http://starship.skyport.net/~just/meta/
> 
> Whether the dotted lines actually actually go through the objects or not is
> left vague on purpous. It depends on whether you'd like to write
>         instance.__class__
>         instance.__turtle__
> or
>         instance->in_class
>         instance->ob_type
> or
>         instance.__class__
>         instance.__class__.__nature__
> or (my fave, so this matches the drawing best)
>         instance.__class__
>         instance.__class__.__class__
> 
> Inheritance is deliberately left out of the picture, since it is defined
> (according to my model) by the metametaclass.
> 
> Does any of this make any sense to anyone?

It does... also looks a lot like what I posted to the egroups
list. I'll just repost it here for simplicity:

                    __class__            __bases__
                        .                    .
        ___________     .    ___________     .    ___________
       |           |    .   |           |    .   |           |
       |     v     |------->|     C     |------->|     B     |
       |___________|        |___________|        |___________|
             |                    |                    |
             |                    |                    |
             |                    |                    |
             v                    v                    v
        ___________          ________________________________
       |           |        |                                |
       | Instance  |        |            Class               |
       |___________|        |________________________________|
             |                              |
             |                              |
             |                              |
             v                              v
        _____________________________________________________
       |                                                     |
       |                    Object                           |
       |_____________________________________________________|
                         |      ^
                         |      |
                         |______| *)


       Down Arrows: __meta__


Arrows to the right indicate delegation which is implemented
by the meta objects (down arrows). Class objects will create
Instance objects when called. The latter are bound to the
Class objects by the __class__ attribute. Class objects
can be bound to each other by specifying a __bases__ tuple.

As opposed to Guido's picture, delegation is defined as abtract
form of behaviour. Instance objects resolve the issue by querying
their __class__ attribute, while Class objects search the
__bases__ tuple.

Note that Class and Instance may also be subclassed allowing
different behaviour to be implemented, e.g. if MyClass were
a subclass of Class and MyInstance a subclass of Instance,
MyClass could create MyInstance objects instead of Instance
objects.

Also note that the protocol used along the __meta__ arrows
has different semantics than along the right arrows: e.g.
__getattr__ is always called.

Here is a sample of the implied protocol:

Say you want to access the attribute 'v.a':

v.a
--> v.__meta__.__getattr__(v,'a')
--> check v.__dict__ for an 'a' entry
--> found: return v.__dict__['a']
--> else: # delegate the request to the Class object
---> v.__class__.__meta__.__getattr__(v.__class__,'a')
---> check v.__class__.__dict__ for 'a'
---> found: return v.__class__.__dict__['a'] (possibly wrapped as
method)
---> else: # delegate the request to the next base object
----> v.__class__.__bases__[0].__meta__.__getattr__(
        v.__class__.__bases__[0],'a')
----> etc.
--> if not found: # call the v.__class__.__getattr__ hook
--> return v.__class__.__getattr__(v,'a')

The above closely describes what the interpreter is doing
in the standard class implementation now.

Note that I left out the meta parts that are handled by the
Object object (a singleton): querying e.g. v.__meta__.__getattr__
would call v.__meta__.__meta__.__getattr__(v.__meta__,'__getattr__')
and so on (until Object is reached).

*) Not sure whether it's a good idea to introduce a loop
here, maybe __meta__ should just be None to indicate the root
of the meta-inheritance tree.

-- 
Marc-Andre Lemburg                               Y2000: 394 days left
---------------------------------------------------------------------
          : Python Pages >>> http://starship.skyport.net/~lemburg/  :
           ---------------------------------------------------------



From just@letterror.com  Wed Dec  2 13:00:02 1998
From: just@letterror.com (Just van Rossum)
Date: Wed, 2 Dec 1998 14:00:02 +0100
Subject: [Types-sig] Re: Meta-classes discussion starter
In-Reply-To: <366508F0.4B896660@lemburg.com>
References: <208801be1c9c$bbdfe100$2d232cd1@linwin.token.hapenney.com>
 <l03102801b288bbd9020d@[193.78.237.175]>
 <l03102802b2897d984c09@[193.78.237.139]>
Message-ID: <l03102802b28adde71871@[193.78.237.134]>

>> +                       PyDict_DelItemString(methods, "__class__");

At 10:31 AM +0100 12/2/98, M.-A. Lemburg wrote:
>I like the patch... but why do you delete the __class__ attribute ?

I must admit that the one aspect of the patch I am not quite confident about...

>To prevent endless recursion in some weird cases ?

No, merely from this view: while playing with the mechanism, I found myself
constantly deleting __class__ from the dict in Python. I could not find any
reason why I would ever need the __class__ entry. It's a little like this:

def somefunction(dict):
    pass

mydict = {"foo": somefunction}

if mydict.has_key("foo"):
    func = mydict["foo"]
    func(mydict)

It just seemed silly to have __class__ still around, especially since I
cannot use it as a real attribute anyway: in instance can only have a Real
Class object as __class__ attribute. So in my experiments I used __klass__
to bind my fake class object to.

[ *** warning: the author is drifting *** ed. ]

*Maybe* it would be even cooler (in today's Python, maybe not 2.0) to use
__metaclass__ (or __turtle__, *whatever*) as the secret word, and have a
second GuidoHook that triggers with that same name *before* it checks for
__class__. Or for my part get rid of the __class__ check altogether. That
way it we could have even more control over classes. Right now, say I
define a little metaclass like this:

class MyMetaClass:
    def __init__(self, name, bases, namesspace):
        ... etc.

Then I can create a "normal" class like this:

class MyClass:
    __class__ = MyMetaClass

This invoked my hook. Then I subclass:

class MySecondClass(MyClass):
    pass

This invokes Guido's hook, which will call MyMetaClass since that's
MyClass's *real* __class__ attribute, so MySecondClass will automatically
be an instance of MyMetaClass. Fine. Almost too perfect.

The caveat is this: Guido's Hook *requires* that __class__ is a real class
(well, no, sure it's not really the hook that forces this, but it's a side
effect of choosing the name __class__ as secret trigger) which means the
constructor of a class is *always* some __init__ func of a metaclass. Which
means we can't emulate the current Python object model in full since we
can't trap access to an "__init__" attr: it will always be found before any
__getattr__ func can get triggered... It really means that I cannot by
definition make this work correctly:

class MyClass:
    __class__ = MyMetaClass
    def __init__(self, a):
        self.a = a

class MySecondClass(MyClass):
    def __init__(self):
        MyClass.__init__(self, 100)

The last line can't work because
- MyClass is in instance of MyMetaClass
- MyClass.__init__ will be found in MyMetaClass *before* it can trigger any
__getattr__ that could look it up inside MyClass.

It's not easy, but it could be made work with Today's Python had "we" not
chosen __class__ as the magic attribute for metahooks...


Just





From just@letterror.com  Wed Dec  2 13:42:04 1998
From: just@letterror.com (Just van Rossum)
Date: Wed, 2 Dec 1998 14:42:04 +0100
Subject: [Types-sig] MetaClasses (Non-ASCII art)
In-Reply-To: <36650AF4.6C6A8519@lemburg.com>
References: <l0310280cb28a106c18a2@[193.78.237.134]>
Message-ID: <l03102803b28aea5704eb@[193.78.237.134]>

At 10:40 AM +0100 12/2/98, M.-A. Lemburg wrote:
>It does... also looks a lot like what I posted to the egroups
>list. I'll just repost it here for simplicity:

Thanks, that is an interesting approach, which actually looks a lot like
the current class/type scheme... The major difference to my (possibly
wrong) view is that I think (actually, hope) your v.__meta__ is the same
object as v.__class__.__class__. So behavior always comes from one class
higher up in the class chain. Maybe this is just dumb. I'll find out...

[ ... ]
>Here is a sample of the implied protocol:
>
>Say you want to access the attribute 'v.a':
>
>v.a

Minor nitpick:

>--> v.__meta__.__getattr__(v,'a')

You can't call it __getattr__, since that means getting an attr from
__meta__. I propose __meta_getattr__. I find Barry's __findattr__
confusing, too, since it reminds me of JPython's use of findattr, which, if
I understood correctly, means something else again...

I'm not sure why inheritance needs te be brought up here (or in any
metaclass discussion) at all: how exactly it works is to be defined by
class.__meta__.

What I'm really saying is, if you define
	getattr(object, attr)
as
	object.__meta__.__meta_getattr__(object, attr)
or as
	object.__class__.__class__.__meta_getattr__(object, attr)
You're basically done. The rest is implementation details... Now, which one
will it be?

Just




From mal@lemburg.com  Wed Dec  2 16:51:31 1998
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 02 Dec 1998 17:51:31 +0100
Subject: [Types-sig] Re: Meta-classes discussion starter
References: <208801be1c9c$bbdfe100$2d232cd1@linwin.token.hapenney.com>
 <l03102801b288bbd9020d@[193.78.237.175]>
 <l03102802b2897d984c09@[193.78.237.139]> <l03102802b28adde71871@[193.78.237.134]>
Message-ID: <36657013.36A4D80C@lemburg.com>

Just van Rossum wrote:
> 
> >> +                       PyDict_DelItemString(methods, "__class__");
> 
> At 10:31 AM +0100 12/2/98, M.-A. Lemburg wrote:
> >I like the patch... but why do you delete the __class__ attribute ?
> 
> I must admit that the one aspect of the patch I am not quite confident about...
> 
> >To prevent endless recursion in some weird cases ?
> 
> No, merely from this view: while playing with the mechanism, I found myself
> constantly deleting __class__ from the dict in Python. I could not find any
> reason why I would ever need the __class__ entry.

I would just leave it there... could become useful in debugging
sessions and AFAI can see, it doesn't do any harm.
 
> [ *** warning: the author is drifting *** ed. ]
[...]
> class MyClass:
>     __class__ = MyMetaClass
>     def __init__(self, a):
>         self.a = a
> 
> class MySecondClass(MyClass):
>     def __init__(self):
>         MyClass.__init__(self, 100)
> 
> The last line can't work because
> - MyClass is in instance of MyMetaClass
> - MyClass.__init__ will be found in MyMetaClass *before* it can trigger any
> __getattr__ that could look it up inside MyClass.

That's why meta objects should have a hook that controlls *all*
getattr-actions, e.g. __meta_getattr__.

-- 
Marc-Andre Lemburg                               Y2000: 394 days left
---------------------------------------------------------------------
          : Python Pages >>> http://starship.skyport.net/~lemburg/  :
           ---------------------------------------------------------



From mal@lemburg.com  Wed Dec  2 16:34:11 1998
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 02 Dec 1998 17:34:11 +0100
Subject: [Types-sig] MetaClasses (Non-ASCII art)
References: <l0310280cb28a106c18a2@[193.78.237.134]> <l03102803b28aea5704eb@[193.78.237.134]>
Message-ID: <36656C03.703AD213@lemburg.com>

Just van Rossum wrote:
> 
> At 10:40 AM +0100 12/2/98, M.-A. Lemburg wrote:
> >It does... also looks a lot like what I posted to the egroups
> >list. I'll just repost it here for simplicity:
> 
> Thanks, that is an interesting approach, which actually looks a lot like
> the current class/type scheme...

Right, it's not much different, only unifies some things by
adding one more level of indirection.

> The major difference to my (possibly
> wrong) view is that I think (actually, hope) your v.__meta__ is the same
> object as v.__class__.__class__. So behavior always comes from one class
> higher up in the class chain. Maybe this is just dumb. I'll find out...

v.__meta__ points to the meta object implementing the
instance behaviour (which is different from class behaviour),
while v.__class__.__meta__ points to a different object
implementing the class behaviour of class C (== v.__class__).

Both meta objects inherit methods and attributes from Object
which would be the root of all things in the Python universe.

> >Here is a sample of the implied protocol:
> >
> >Say you want to access the attribute 'v.a':
> >
> >v.a
> 
> Minor nitpick:
> 
> >--> v.__meta__.__getattr__(v,'a')
> 
> You can't call it __getattr__, since that means getting an attr from
> __meta__.

Agreed. It is a bit confusing, especially since that __getattr__
is always called and doesn't really implement the usual __getattr__
behaviour (which would be getting an attribute from v.__meta__ as
you correctly stated).

> I propose __meta_getattr__. I find Barry's __findattr__
> confusing, too, since it reminds me of JPython's use of findattr, which, if
> I understood correctly, means something else again...

That's a good idea: all special hooks needed could be given a
'meta_' prefix to indicate that these work in different ways
than the standard ones.

> I'm not sure why inheritance needs te be brought up here (or in any
> metaclass discussion) at all: how exactly it works is to be defined by
> class.__meta__.

Inheritance for meta objects can be very useful, e.g. in case
all you want is to refine say the __meta_getattr__ method of
Class to enable automatic acquisition or caching. You won't
have to copy all the other implementation details: inheritance
does this for you.

> What I'm really saying is, if you define
>         getattr(object, attr)
> as
>         object.__meta__.__meta_getattr__(object, attr)
> or as
>         object.__class__.__class__.__meta_getattr__(object, attr)
> You're basically done. The rest is implementation details... Now, which one
> will it be?

Not really: classes and instances do different things to get
at their attributes. Instances revert to their classes for help
if they can't find an attribute, while classes try their
base classes. This means two different implementations, so
v.__meta__.__meta_getattr__ will in fact call getattr(v.__class__,name)
which is implemented by v.__class__.__meta__.__meta_getattr__.

-- 
Marc-Andre Lemburg                               Y2000: 394 days left
---------------------------------------------------------------------
          : Python Pages >>> http://starship.skyport.net/~lemburg/  :
           ---------------------------------------------------------




From JOrendorff@ixl.com  Wed Dec  2 18:58:13 1998
From: JOrendorff@ixl.com (JOrendorff@ixl.com)
Date: Wed, 02 Dec 1998 10:58:13 -0800
Subject: [Types-sig] MetaClasses (Non-ASCII art)
References: <l0310280cb28a106c18a2@[193.78.237.134]> <l03102803b28aea5704eb@[193.78.237.134]> <36656C03.703AD213@lemburg.com>
Message-ID: <36658DC5.58661548@ixl.com>

> That's a good idea: all special hooks needed could be given a
> 'meta_' prefix to indicate that these work in different ways
> than the standard ones.

Wait a minute.  Why are we doing this again?  :-(

I thought we were going to unify classes and types, not just give types
a new name and fancy hooks.  :-(

-- 
Jason
confused little turtle
likes Guido's ascii art


From da@skivs.ski.org  Wed Dec  2 17:59:10 1998
From: da@skivs.ski.org (David Ascher)
Date: Wed, 2 Dec 1998 09:59:10 -0800 (Pacific Standard Time)
Subject: [Types-sig] MetaClasses (Non-ASCII art)
In-Reply-To: <l03102803b28aea5704eb@[193.78.237.134]>
Message-ID: <Pine.WNT.4.04.9812020956460.197-100000@rigoletto.ski.org>

On Wed, 2 Dec 1998, Just van Rossum wrote:

> At 10:40 AM +0100 12/2/98, M.-A. Lemburg wrote:
> >It does... also looks a lot like what I posted to the egroups
> >list. I'll just repost it here for simplicity:
> 
> Thanks, that is an interesting approach, which actually looks a lot like
> the current class/type scheme... 

Both of you (MAL and JvR) present schemes in which the notion of class and
instance (and something else, call it metaclass) are reified.  This is in
contrast, I believe, to GvR's view, which allows unlimited 'levels of
metaness', aka turtledepth.

This may be a good thing, or not.  I'm not sure.  Smalltalk, I believe,
also has only three levels.  Does it make sense to have arbitrary
turtledepth?

--david



From mal@lemburg.com  Wed Dec  2 18:45:24 1998
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 02 Dec 1998 19:45:24 +0100
Subject: [Types-sig] MetaClasses (Non-ASCII art)
References: <Pine.WNT.4.04.9812020956460.197-100000@rigoletto.ski.org>
Message-ID: <36658AC4.16AA2A71@lemburg.com>

David Ascher wrote:
> 
> On Wed, 2 Dec 1998, Just van Rossum wrote:
> 
> > At 10:40 AM +0100 12/2/98, M.-A. Lemburg wrote:
> > >It does... also looks a lot like what I posted to the egroups
> > >list. I'll just repost it here for simplicity:
> >
> > Thanks, that is an interesting approach, which actually looks a lot like
> > the current class/type scheme...
> 
> Both of you (MAL and JvR) present schemes in which the notion of class and
> instance (and something else, call it metaclass) are reified.  This is in
> contrast, I believe, to GvR's view, which allows unlimited 'levels of
> metaness', aka turtledepth.

Can't speak for Just, but in the scheme I have in mind there
can be any number of objects between Object and the top level
objects.

I just left them out to avoid confusion (see my note at the bottom
of the drawing).
 
> This may be a good thing, or not.  I'm not sure.  Smalltalk, I believe,
> also has only three levels.  Does it make sense to have arbitrary
> turtledepth?

Well, I think practically speaking there's probably not much use
for arbitrary depth in meta-structures.

Just to clarify this a little: I'm talking about real meta levels
here not inheritance levels. Meta levels are always recursed into
while inheritance does not enforce this.

-- 
Marc-Andre Lemburg                               Y2000: 394 days left
---------------------------------------------------------------------
          : Python Pages >>> http://starship.skyport.net/~lemburg/  :
           ---------------------------------------------------------



From just@letterror.com  Thu Dec  3 02:19:21 1998
From: just@letterror.com (Just van Rossum)
Date: Thu, 3 Dec 1998 03:19:21 +0100
Subject: [Types-sig] MetaClasses (Non-ASCII art)
In-Reply-To: <36656C03.703AD213@lemburg.com>
References: <l0310280cb28a106c18a2@[193.78.237.134]>
 <l03102803b28aea5704eb@[193.78.237.134]>
Message-ID: <l03102804b28ba0fa9f9f@[193.78.237.133]>

Me:
>> I'm not sure why inheritance needs te be brought up here (or in any
>> metaclass discussion) at all: how exactly it works is to be defined by
>> class.__meta__.

M.-A. Lemburg wrote:
>Inheritance for meta objects can be very useful, e.g. in case
>all you want is to refine say the __meta_getattr__ method of
>Class to enable automatic acquisition or caching. You won't
>have to copy all the other implementation details: inheritance
>does this for you.

Sure, just add one more meta level which just implements
inheritance and you're still done...

Me:
>> What I'm really saying is, if you define
>>         getattr(object, attr)
>> as
>>         object.__meta__.__meta_getattr__(object, attr)
>> or as
>>         object.__class__.__class__.__meta_getattr__(object, attr)
>> You're basically done. The rest is implementation details... Now, which one
>> will it be?

M.-A. Lemburg wrote:
>Not really: classes and instances do different things to get
>at their attributes. Instances revert to their classes for help
>if they can't find an attribute, while classes try their
>base classes.

Sure, but they're _both_ implemented as classes, and they
just happen to define __meta_getattr__ differently. I don't see
how this has anything to do with inheritance. The class protocol
defines inheritance, but might be implemented using a meta
protocol which doesn't do inheritance. Or one that does, but that
one might be implemented using a meta protocol which doesn't
do inheritance. Or one that does, but that one might be
implemented using a meta protocol which doesn't do inheritance.
Or one that does, but that one might be implemented using a
meta protocol which doesn't do inheritance.

;-)

>This means two different implementations, so
>v.__meta__.__meta_getattr__ will in fact call getattr(v.__class__,name)
>which is implemented by v.__class__.__meta__.__meta_getattr__.

Yes, and? ;-)


Just




From just@letterror.com  Thu Dec  3 01:57:42 1998
From: just@letterror.com (Just van Rossum)
Date: Thu, 3 Dec 1998 02:57:42 +0100
Subject: [Types-sig] Re: Meta-classes discussion starter
In-Reply-To: <36657013.36A4D80C@lemburg.com>
References: <208801be1c9c$bbdfe100$2d232cd1@linwin.token.hapenney.com>
 <l03102801b288bbd9020d@[193.78.237.175]>
 <l03102802b2897d984c09@[193.78.237.139]>
 <l03102802b28adde71871@[193.78.237.134]>
Message-ID: <l03102803b28b9f934b1d@[193.78.237.133]>

>> The last line can't work because
>> - MyClass is in instance of MyMetaClass
>> - MyClass.__init__ will be found in MyMetaClass *before* it can trigger any
>> __getattr__ that could look it up inside MyClass.

M.-A. Lemburg wrote:
>That's why meta objects should have a hook that controlls *all*
>getattr-actions, e.g. __meta_getattr__.

Yeah, they _should_, but I'm talking about this from the perspective of
trying to implement such a thing using Python _today_. I get a long way by
defining an alternative __dict__ and the usual __{get|set|del}attr__ cruft,
but this was a (veryveryvery*very* minor) point where today's Python
implementation let me down...


Just




From just@letterror.com  Thu Dec  3 01:52:58 1998
From: just@letterror.com (Just van Rossum)
Date: Thu, 3 Dec 1998 02:52:58 +0100
Subject: [Types-sig] MetaClasses (Non-ASCII art)
In-Reply-To: <36658AC4.16AA2A71@lemburg.com>
References: <Pine.WNT.4.04.9812020956460.197-100000@rigoletto.ski.org>
Message-ID: <l03102802b28b97635df2@[193.78.237.133]>

>David Ascher wrote:
>> Both of you (MAL and JvR) present schemes in which the notion of class and
>> instance (and something else, call it metaclass) are reified.  This is in
>> contrast, I believe, to GvR's view, which allows unlimited 'levels of
>> metaness', aka turtledepth.

M.-A. Lemburg wrote:
>Can't speak for Just, but in the scheme I have in mind there
>can be any number of objects between Object and the top level
>objects.
>
>I just left them out to avoid confusion (see my note at the bottom
>of the drawing).

Same here: I showed a possible hierarchy that would (if it could!)
implement a system which is semantically very close to what we have now.
What I'm really saying is:

	an object's implementation lives in it's class
	how the two interact is defined by it's class's class

To use the current Python object model as an example as how this could work:

	an instance's implementation lives in it's __class__
	how one accesses attributes of object is defined by
	object.__class__.__class__ (ie. search ob.__dict__, then ask class)

	class = object.__class__

	class's implementation lives in it's __class__
	how one accesses attributes from class is defined by
	class.__class__.__class__ (ie. search class.__dict__, then
	ask each of class.__bases__)

(to turn this into Marc-André's scheme:
     print string.replace(justs_scheme, "__class__.__class__", "__meta__")
;-)

Or, in other words:

	the instance protocol is implemented by:
		instance.__class__.__class__
	the class protocol is implemented by
		instance.__class__.__class__.__class__


What I hope to find out is what I'm saying is 1) possible, 2) makes sense...

So, to reimplement Python's notion of instances and classes using "my" meta
protocol you only _need_ 4 levels, but noone will stop you to invent a few
more...

Just




From tim_one@email.msn.com  Thu Dec  3 02:49:05 1998
From: tim_one@email.msn.com (Tim Peters)
Date: Wed, 2 Dec 1998 21:49:05 -0500
Subject: [Types-sig] Re: Meta-classes discussion starter
In-Reply-To: <1.5.4.32.19981202081755.00f58f34@triode.net.au>
Message-ID: <000201be1e67$7e0b7f20$fb9e2299@tim>

[John Skaller]
> ...
> And that is it. :-) There is no such thing as a class :-))
> It is nothing more than a function which makes objects.

[Tim]
>> John, Python is not a functional language, and there's not a shadow
>> of a hint that it will ever become one.

[John]
>         You're not following.

You're not anticipating <wink>.

> I'm analysing the _structure_ of a system. I'm telling you classes are
> _isomorphic_ to functions; and providing the isomorphism.

Up above, yes; below, no.  You went quite a ways beyond casting the
semantics in functional form:  (A) you want others to adopt that view of
semantics too; and, (B) you want programmatic support for that view added to
Python.

> I didn't claim 'python was a functional language'.

No, I hardly think anyone would.  You do want to move it in that direction,
though, don't you?  Let's cut to the chase:

> ...
> My suggestion is for more builtin functions, of the kind needed
> to construct functions.

>> How about a specific proposal?

>         There is some code available for browsing.

How about a specific proposal?

most-people-don't-even-get-one-engraved-invitation<wink>-ly y'rs  - tim




From mal@lemburg.com  Thu Dec  3 09:43:05 1998
From: mal@lemburg.com (M.-A. Lemburg)
Date: Thu, 03 Dec 1998 10:43:05 +0100
Subject: [Types-sig] MetaClasses (Non-ASCII art)
References: <l0310280cb28a106c18a2@[193.78.237.134]>
 <l03102803b28aea5704eb@[193.78.237.134]> <l03102804b28ba0fa9f9f@[193.78.237.133]>
Message-ID: <36665D29.2636BA03@lemburg.com>

Just van Rossum wrote:
> 
> Me:
> >> I'm not sure why inheritance needs te be brought up here (or in any
> >> metaclass discussion) at all: how exactly it works is to be defined by
> >> class.__meta__.
> 
> M.-A. Lemburg wrote:
> >Inheritance for meta objects can be very useful, e.g. in case
> >all you want is to refine say the __meta_getattr__ method of
> >Class to enable automatic acquisition or caching. You won't
> >have to copy all the other implementation details: inheritance
> >does this for you.
> 
> Sure, just add one more meta level which just implements
> inheritance and you're still done...

Ok, point taken, would cause a lot of overhead though...

> Me:
> >> What I'm really saying is, if you define
> >>         getattr(object, attr)
> >> as
> >>         object.__meta__.__meta_getattr__(object, attr)
> >> or as
> >>         object.__class__.__class__.__meta_getattr__(object, attr)
> >> You're basically done. The rest is implementation details... Now, which one
> >> will it be?
> 
> M.-A. Lemburg wrote:
> >Not really: classes and instances do different things to get
> >at their attributes. Instances revert to their classes for help
> >if they can't find an attribute, while classes try their
> >base classes.
> 
> Sure, but they're _both_ implemented as classes, and they
> just happen to define __meta_getattr__ differently. I don't see
> how this has anything to do with inheritance. The class protocol
> defines inheritance, but might be implemented using a meta
> protocol which doesn't do inheritance. Or one that does, but that
> one might be implemented using a meta protocol which doesn't
> do inheritance. Or one that does, but that one might be
> implemented using a meta protocol which doesn't do inheritance.
> Or one that does, but that one might be implemented using a
> meta protocol which doesn't do inheritance.

Right. I just find it a little confusing searching for the
meta object (implementing the object's behaviour) in
obj.__class__.__class__ because that reminds me too much of
how Instances and Classes relate to each other. 

The meta protocol could very well also be used for other objects
types which don't rely on underlying Classes. Coming to think
of it... these __meta__ object have a lot in common with the
type objects: only that type objects are a symbiosis of behaviour
*and* implementation. Maybe there's a possibility there...

Also, it could allow changing the behaviour by reassigning a new
meta object to obj.__meta__ on a per object basis.

And last, not least, it avoids one level of indirection
in the implementation (though this could probably be simulated
by caching __class__.__class__ into the __meta__ attribute).

> >> The last line can't work because
> >> - MyClass is in instance of MyMetaClass
> >> - MyClass.__init__ will be found in MyMetaClass *before* it can trigger any
> >> __getattr__ that could look it up inside MyClass.
> 
> M.-A. Lemburg wrote:
> >That's why meta objects should have a hook that controlls *all*
> >getattr-actions, e.g. __meta_getattr__.
> 
> Yeah, they _should_, but I'm talking about this from the perspective of
> trying to implement such a thing using Python _today_. I get a long way by
> defining an alternative __dict__ and the usual __{get|set|del}attr__ cruft,
> but this was a (veryveryvery*very* minor) point where today's Python
> implementation let me down...

I wouldn't be afraid of thinking big: a rewrite of some of the
stuff in classobject.c wouldn't hurt much and there's still a
lot of time 'til 1.6 comes out ;-)

-- 
Marc-Andre Lemburg                               Y2000: 393 days left
---------------------------------------------------------------------
          : Python Pages >>> http://starship.skyport.net/~lemburg/  :
           ---------------------------------------------------------




From skaller@maxtal.com.au  Thu Dec  3 11:25:43 1998
From: skaller@maxtal.com.au (John Skaller)
Date: Thu, 03 Dec 1998 21:25:43 +1000
Subject: [Types-sig] Re: Meta-classes discussion starter
Message-ID: <1.5.4.32.19981203112543.00f61e60@triode.net.au>

At 21:49 2/12/98 -0500, Tim Peters wrote:
>[John]
>>         You're not following.
>
>You're not anticipating <wink>.

        I have to pay that :-)

>> I'm analysing the _structure_ of a system. I'm telling you classes are
>> _isomorphic_ to functions; and providing the isomorphism.
>
>Up above, yes; below, no.  You went quite a ways beyond casting the
>semantics in functional form:  (A) you want others to adopt that view of
>semantics too; 

        No, I don't want other to adopt it, I want them to use
it as an analysis tool, to understand what meta-programming is all 
about.

        If one were to assume classes were functions and
do a functional analysis of what meta-programming is
from that viewpoint, one might be able to take the results
and map them _back_ onto a class model to get meta-classes.

>and, (B) you want programmatic support for that view added to
>Python.

        I want some specific functions added to builtins,
because (a) I need them and (b) I know that they're
enough to do all meta-programming. I have theorems that prove
it. I have constructions for all the common data structures
in a book, including stacks, queues, iterators, arrays.
[RFC Walters, Categories in Computer Science]

>> I didn't claim 'python was a functional language'.
>
>No, I hardly think anyone would.  You do want to move it in that direction,
>though, don't you?  

        No, I don't. I want to move in the direction a categorical
programming system would take. That requires throwing out
encapsulation (which Python doesn't enforce anyhow) and emphasising
second order constructions [operations on function] rather than
first order ones [operations on data structures]. Because that is
what 'meta' means.

        It's quite clear this is what meta-classes are about:
defining how classes define instances.

        And the 'class is a function' view shows, in the abstract,
what is required IMHO: you have to be able to construct functions dynamically.

>> My suggestion is for more builtin functions, of the kind needed
>> to construct functions.
>
>>> How about a specific proposal?

        I would not waste my time without some support.
I have learned this from 5 years on ISO Standardisation
committees. People against it will argue every which way
against it, and without an implementation it isn't possible
to provide a demonstration of utility.

        I am in fact constructing these objects:
see the 'felix' subsystem of interscript.

        I am writing here only to try to prevent
adding yet another hack that doesn't quite work to
a programming language. See C++. I don't want that
to happen to Python: it has enough non-orthogonal
implementation hacks -- such as classes -- already.

>>         There is some code available for browsing.
>
>How about a specific proposal?

        Sure:

        compose(f1, f2, f3, f4 ..) # chain in series, 1 arg only
        product(f1, f2, f3, f4, ..) # apply componentwise (in parallel)
        sum(f1, f2, .. ) # select one of
        delta(x,i) = (x,x,x,x,x ..) # i times

there are a few more. I will have to look up the book.
Sum needs explanation: it is called like

        f(x, i) # return fi(x)

There is also a case to expand the set of available
quantifiers. (reduce is a quantifier, i.e. a loop)

Most of these constructions are available in C++ STL.
They're the basis of (generic) programming.

The constructions above are precisely what is needed to
build all flowcharts. if you can do that, you can
build _any_ function dynamically, given some suitable
primitives to combine.

So I am not asking for any major changes, I'd like a
basis: i.e. a set of constructions spanning the
whole space. Given those, new more convenient ones
can be defined, and the most used of these can
be re-implemented later in C when it is known
which ones are used most, or need the most speed
improvements.

For example, to compose two functions, one might
actually concatenate the opcodes.

IF you have these facilities available, you can actually
start talking about implementing meta-classes. if you really
want to. After all, aren't they meant to be a _general_ way to
define something that defines a class? And what is that, other
than a way of writing a 'mini' compiler which places an
arbitrary interpretation on a class definition?
And isn't that minicompiler just a function?
-------------------------------------------------------
John Skaller    email: skaller@maxtal.com.au
		http://www.maxtal.com.au/~skaller
		phone: 61-2-96600850
		snail: 10/1 Toxteth Rd, Glebe NSW 2037, Australia




From just@letterror.com  Thu Dec  3 14:02:13 1998
From: just@letterror.com (Just van Rossum)
Date: Thu, 3 Dec 1998 15:02:13 +0100
Subject: [Types-sig] MetaClasses (Non-ASCII art)
In-Reply-To: <36665D29.2636BA03@lemburg.com>
References: <l0310280cb28a106c18a2@[193.78.237.134]>
 <l03102803b28aea5704eb@[193.78.237.134]>
 <l03102804b28ba0fa9f9f@[193.78.237.133]>
Message-ID: <l03102801b28c43c38ee1@[193.78.237.179]>

Me:
>> Sure, just add one more meta level which just implements
>> inheritance and you're still done...

M.-A. Lemburg wrote:
>Ok, point taken, would cause a lot of overhead though...

Absolutely. But we're having a theoretical discussion anyway, trying not to
think too much about actual implementation...

Me:
>> Or one that does, but that one might be implemented using a
>> meta protocol which doesn't do inheritance.

M.-A. Lemburg wrote:
>Right. I just find it a little confusing searching for the
>meta object (implementing the object's behaviour) in
>obj.__class__.__class__ because that reminds me too much of
>how Instances and Classes relate to each other.

Well, in the (maybe: "my") meta sense they are the same: a class is an
instance of a metaclass. But in the Python sense they are indeed very
different: getting a function attribute from a Python instance may look
inside it's __class__ and return a bound method, while getting a function
attribute from a class will always return an unbound method and never even
searches class.__class__. This is exactly the kind of behavior that one
would like to express as some sort of metathingy or turtle, and therefore
only serves as an _example_ of what you could do with a meta protocol. If
we want to be able to express these things, we should try to forget how
current Python object semantics are and come up with something _simple_
that is above it all. Or something.

>The meta protocol could very well also be used for other objects
>types which don't rely on underlying Classes. Coming to think
>of it... these __meta__ object have a lot in common with the
>type objects: only that type objects are a symbiosis of behaviour
>*and* implementation. Maybe there's a possibility there...

And we're really back to square one: types *and* classes...

>Also, it could allow changing the behaviour by reassigning a new
>meta object to obj.__meta__ on a per object basis.

Yeah, maybe that's cool. But then again: types *and* classes...

>And last, not least, it avoids one level of indirection
>in the implementation (though this could probably be simulated
>by caching __class__.__class__ into the __meta__ attribute).

Right: implementation details!

>> but this was a (veryveryvery*very* minor) point where today's
>> Python implementation let me down...
>
>I wouldn't be afraid of thinking big: a rewrite of some of the
>stuff in classobject.c wouldn't hurt much and there's still a
>lot of time 'til 1.6 comes out ;-)

I don't think that's worth it: this is all just a thought experiment, and I
think we get a long way implementing all sorts of experiments using the
simple hooks we have. Once one of these experiments is successful, it can
start having it's influence on classobject.c. And I truly think that's a
matter of 2.0.

MetaJust





From Edward Welbourne <eddy@chaos.org.uk>  Thu Dec  3 19:54:53 1998
From: Edward Welbourne <eddy@chaos.org.uk> (Edward Welbourne)
Date: Thu, 3 Dec 1998 19:54:53 GMT
Subject: [Types-sig] types, functional stuff.
Message-ID: <199812031954.TAA29203@lsls4p.lsl.co.uk>

A quick suggestion on the `how to specify static typing' debate ... then
a little contribution on functional tools.

Once upon a time, so I'm told, folk started putting

__doc__ = """The documentation of this entity"""

at the starts of entities, and it was seen to be good.

OK, so a function (or method) can start with

def myfunc(first, flexible, optional=7):
    __statictypingdata__ = {
	'': types.IntegerType,	# The return type
	'first': types.FloatType,	# The type of this arg
	'flexible': (types.StringType, types.TupleType, types.ListType),
	# that's offering a choice
	'optional': (None, types.IntType),	# NB None != NoneType
	# and there's nothing to stop us putting in info about local
	# variables, too.
	}

and a class implementing .__call__() could do similar.
The big problem is specifying that a dictionary argument is to be a
dictionary mapping integers to objects of class Hubert.
This just seems much more practical than whacky syntax.


Now, John Skaller asked for:

>        compose(f1, f2, f3, f4 ..) # chain in series, 1 arg only
>        product(f1, f2, f3, f4, ..) # apply componentwise (in parallel)
>        sum(f1, f2, .. ) # select one of
>        delta(x,i) = (x,x,x,x,x ..) # i times

so, John: there follow implementations for 3 of these in python (you can
do currying too, it's not hard).  Use them.  If the uses persuade folk
they are useful, maybe someone'll bother to implement them as a C
module.  The language doesn't need them as built-ins.  I haven't done
sum: it would involve working out what types the functions f1, ... take
as their arguments and I don't have time or the right docs to hand.
But hey, I'll throw in a close relative (union).

def delta(what, count=2): return (what,) * count

class compose:
	def __init__(self, *args):
		row = []
		# this will be in reverse order so that
		# compose(f1,f2)(x) = f1(f2(x))
		for item in args:
			if isinstance(item, _Composite):
				# gratuitous optimisation
				row = item.__components + row
			else:
				row[:0] = [ item ]

		self.__components = row

	def __call__(self, *args):
		for item in self.__components:
			args = apply(item, args)
			if type(args) != types.TupleType: args = (args,)

		return args
		# yeah, sorry about returning a tuple always.

class product:
	def __init__(self, *args):
		self.__factors = args

	def __call__(self, *args):
		if len(args) != len(self.__factors):
			raise TypeError, 'Wrong number of arguments'

		return map(lambda f,x: f(x), self.__factors, args)

class union:
	def __init__(self, parts):
		"""parts is a dict, mapping types to functions"""
		self.__parts = parts

	def __call__(self, what):
		try: part = self.__parts(type(what))
		except KeyError:
			raise TypeError, 'Argument not supported by this direct sum'
		return part(what)

and no, I haven't tested them: I just typed them out of my head.  The
basic lesson is that python is a perfectly good implementation domain
for all the functional things I ever want.  They practically write
themselves, so don't complain about their absence from the language.
(But lambda is a necessity !)

I'm not subscribing to this list, just reading web archive when I have time.
So don't be offended if I'm slow responding to any replies ;^)
Dig the crazy meta-stuff, but it confuses me !

	Eddy.


From evan@tokenexchange.com  Thu Dec  3 20:10:03 1998
From: evan@tokenexchange.com (Evan Simpson)
Date: Thu, 3 Dec 1998 14:10:03 -0600
Subject: [Types-sig] Q: What are Meta-classes anyhoo?
Message-ID: <28da01be1ef8$ea915bb0$2d232cd1@linwin.token.hapenney.com>

I meant to keep up with the the discussion following my overly-bold
meta-class post (provoking discussion was my intent, and it has been amply
fulfilled), but have been ill and cut off from the Internet for two days.
Now I've read it all, and I'm going to dump mental core all over the place.

In my response to GvR's ASCII repost, I realized that I did not have a
decent grasp of what other folks were proposing.  Now I don't even know what
*I* mean by a meta-class any more.

Just van Rossum, Don Beaudry, and MA Lemburg all seem to have roughly the
same idea of what meta-classes would be, and what they would be *for*, but
this idea isn't made explicit, at least enough for my poor brain to get a
handle on it.

John Skaller seems to have a very different take, viewing meta-classes as a
tool for meta-programming.  This seems to involve the ability to make class
factories, which would construct complex functions from simple functions by
much more regular and analytically sound methods than simply compiling a
string.

I'm not sure where Tim Peters stands, except that John's approach really
bothers him <wink>.

As I said, I'm dumping core here, so here's a list of my thoughts and
questions in no particular order:

1. We really want to do away with types vs. classes, and making types *be*
classes is the general favorite approach.  Since every object has a type,
this means that every object will have a class.  Including classes (thus
meta-class).

2. Types are inert lumps, good only for identity comparisons with other
types.  Classes provide default attributes and methods to their instances.
Classes *get* default attributes and methods from their base classes, in a
very powerful and useful way.  What should they get from their meta-class?

3. JvR+DB+MAL, if I'm reading them right, have meta-classes defining or
possibly subverting what it means for an object to be a class or an
instance.  Am I reading this right?

4. For example, MAL has the meta-object "Instance" of an instance (which is
*not* its class!) control how the instance finds attributes outside its
dictionary.

5. JvR would have the class of the class of the instance take this meta-
role.  Similarly, the class of the class of a class would define attribute
search and instance creation for the class <head: boom>

6. DB, on the other hand, seems to suggest in one message that a meta-class'
methods might merely define static methods of its classes, just as class
methods define static (read "bound") methods of its instances.  I sort of
took this view initially.

7. Since attribute search behavior is the only example I've seen of the
meta- role's functionality, it's not clear to me how this is superior to
sub-classing with a mixin.  Mr. Lemburg's __meta__ stuff in particular
bemuses and befuddles me, since it seems to have little or nothing to do
with point #1.

8. Another point of view (Mr. Skaller's, I think) is that class objects
build instance objects, albeit in a fairly passive way.  Thus, meta-classes
should build classes, but in a more active fashion than simply acting as a
template/default-attribute map (which base classes do anyhow).  I have vague
visions of filling in class methods by combining sub-function code snippets
in interesting ways.

9. Might this be too static, since changing a meta-class would be like
changing a class' __init__ rather than like changing its dictionary?  That
is, the meta-class is only executed to build the class, not dynamically
searched.

10. There is no point #10.



From jorendor@cbu.edu  Thu Dec  3 22:16:06 1998
From: jorendor@cbu.edu (jorendor@cbu.edu)
Date: Thu, 03 Dec 1998 14:16:06 -0800
Subject: [Types-sig] optional typing and performance
References: <199812012144.QAA18669@p5mts.nielsenmedia.com>
Message-ID: <36670DA6.345DF264@ixl.com>

> The actual format for type declaration does not matter to me
>  as long as parser/compiler can recognize it.

Well, I'm going to throw my ear into the ring here..

I don't want to say ''' ''' every time I want to add type info.

According to what I've heard, Guido doesn't intend for Python 2.x
code to run under Python 1.x-- or indeed vice versa.  So "degrade
gracefully" isn't among the design criteria.

...

My preferred syntax looks more like C:

  def f(str text, int start):
      re.MatchObject match = _bi_re.search(text, start)
      if match is None:
          raise fError("not found")
      else:
          int end = match.end()
          return text[start:end]

The compiler should figure out the type info expressions
('str', 'int', 're.MatchObject') as soon as names have been bound.
If, for some reason, one of them can't be bound to a Type (Class?)
object at the moment, it can generate bytecode for a runtime check.

I don't like the idea of using ':' to delimit Python type info.
The colon is already used to mean a few other things.

The above is, IMHO, as readable as the rest of Python.  Anything
more goes into the verbose.

I think the builtins 'int', 'float', 'long' and 'str'
(and maybe 'tuple' and 'list') should become Class (Type?)
objects in Python2, so you can do this:

  if isinstance(x, str):  f(x, 0)

Well, I've rambled long enough.  I was working on some major
metaclassing revelations this morning, but my head exploded.

-- 
Jason


From vorpal@xs4all.nl  Thu Dec  3 21:22:34 1998
From: vorpal@xs4all.nl (Niels Ferguson)
Date: Thu, 03 Dec 1998 22:22:34 +0100
Subject: [Types-sig] Why have two hierarchies?
In-Reply-To: <28da01be1ef8$ea915bb0$2d232cd1@linwin.token.hapenney.com>
Message-ID: <3.0.3.32.19981203222234.0093ccc0@pop.xs4all.nl>

I have been trying to follow the meta-class discussion (with varying
success), so it is quite possible that my comments are due to my limited
understanding of the subject.

I find the idea of having a meta-class hierarchy and a separate class
derivation hierarchy confusing. My question is whether you cannot merge the
two into a single framework.

The idea is to make the meta-class the first class in the list of
parent-classes. To allow the meta-class functions to do their work, all we
need to do is to provide access to the list of the rest of the subclasses
in some way. This gives the meta-class the first chance to intercept any
method access, and implement any behaviour that it likes. In particular, it
can control how the behaviour of the class depends on the subclasses. 

I don't have the overview over the Python internals to know how this would
have to be implemented. It requires some form of a generalised __getitem__
mechanism. However, this seems simpler (particularly in definition) than
introducing a second hierarchy.

Niels



============================================================
Niels Ferguson, vorpal@xs4all.nl, voice/fax: +31 20 463 0977
PGP: 50E0 CBE2 3F19 C17D  FBE3 FFA5 38B7 BBB5


From gmcm@hypernet.com  Thu Dec  3 21:47:37 1998
From: gmcm@hypernet.com (Gordon McMillan)
Date: Thu, 3 Dec 1998 16:47:37 -0500
Subject: [Types-sig] Q: What are Meta-classes anyhoo?
In-Reply-To: <28da01be1ef8$ea915bb0$2d232cd1@linwin.token.hapenney.com>
Message-ID: <1299418267-3638894@hypernet.com>

[Evan regrets opening his big mouth, threatens to change his name to 
Thelma and move to Polynesia...]

> I meant to keep up with the the discussion following my overly-bold
> meta-class post (provoking discussion was my intent, and it has been
> amply fulfilled), but have been ill and cut off from the Internet
> for two days. Now I've read it all, and I'm going to dump mental
> core all over the place.

Well at least you didn't have to scrape it off the walls and send it 
to us!

> 1. We really want to do away with types vs. classes, and making
> types *be* classes is the general favorite approach.  Since every
> object has a type, this means that every object will have a class. 
> Including classes (thus meta-class).

By itself, it just means things like ints and floats are classes. The 
fact that classes are likely to have classes is toxic gravy.

> 2. Types are inert lumps, good only for identity comparisons with
> other types.  Classes provide default attributes and methods to
> their instances. Classes *get* default attributes and methods from
> their base classes, in a very powerful and useful way.  What should
> they get from their meta-class?

Oh no! Types have methods and state, it's just that they are only 
minimally manipulable from pure Python.

> 3. JvR+DB+MAL, if I'm reading them right, have meta-classes defining
> or possibly subverting what it means for an object to be a class or
> an instance.  Am I reading this right?

Yes. But there's lots of room for argument about what is meant by 
"subvert", "class", "instance" and even "is".

> 4. For example, MAL has the meta-object "Instance" of an instance
> (which is
> *not* its class!) control how the instance finds attributes outside its
> dictionary.
> 
> 5. JvR would have the class of the class of the instance take this
> meta- role.  Similarly, the class of the class of a class would
> define attribute search and instance creation for the class <head:
> boom>
> 
> 6. DB, on the other hand, seems to suggest in one message that a
> meta-class' methods might merely define static methods of its
> classes, just as class methods define static (read "bound") methods
> of its instances.  I sort of took this view initially.

Here, I think, you're running into the difference between the 
"theoretical" view and the "practical" view. In the former, it's all 
important that everything be an object, and that all objects be 
instances of classes, recursing to a Deus Ex Machina. However, even 
the venerable (and quite mad) Don Beaudry, who created the hook in 
the first place and went furthest in making a theoretically pure 
implementation, admits that he has no idea what this conceptual 
purity buys you, other than the ability to be a sanctimonious madman.

The "practical" side is concerned with doing __getattr__ and 
__setattr__ hooks in a manner that doesn't cause people to confuse 
you with Don Beaudry.

GvR's ASCII art and MAL's proposal are firmly in the latter camp - 
separate the machinery from the attributes. Just's is too, even 
though it looks more and more Don-ish all the time.

(A naive observer might expect Tim to favor the former, as it has no 
known usefulness.  But I suspect he's sees more ability to subvert 
usefulness in the latter.)

> 7. Since attribute search behavior is the only example I've seen of
> the meta- role's functionality, it's not clear to me how this is
> superior to sub-classing with a mixin.  Mr. Lemburg's __meta__ stuff
> in particular bemuses and befuddles me, since it seems to have
> little or nothing to do with point #1.

Subclassing with a mixin doesn't let you, for example, interfere with 
how an existing attribute is accessed. The general idea here is to 
kidnap the object, skin it, then waltz around in public impersonating 
it. All without letting the programmer / user know he's been 
bamboozled.

> 8. Another point of view (Mr. Skaller's, I think) is that class
> objects build instance objects, albeit in a fairly passive way. 
> Thus, meta-classes should build classes, but in a more active
> fashion than simply acting as a template/default-attribute map
> (which base classes do anyhow).  I have vague visions of filling in
> class methods by combining sub-function code snippets in interesting
> ways.

John speaks exotically, but the idea is much the same. Meta-classes 
create classes, but what we're looking for is ways to change the 
class plumbing. Who says a class has to hold it's attributes in a 
dict? Maybe it holds them as slots for faster access, but you have to 
promise that the table of slots is fixed at __init__ time. Then the 
meta-object is in charge of making sure that other objects don't have 
to know that this class is implemented very differently.
 
> 9. Might this be too static, since changing a meta-class would be
> like changing a class' __init__ rather than like changing its
> dictionary?  That is, the meta-class is only executed to build the
> class, not dynamically searched.

The point is that it's in charge of how the search is performed. 
There's nothing static about it, or at least there won't be unless 
Roger breaks out of the isolation ward.

> 10. There is no point #10.

Good, I was running out of people to offend!

metaclasses-for-the-metaclueless-ly y'rs

- Gordon


From retnuh@pixar.com  Thu Dec  3 22:33:16 1998
From: retnuh@pixar.com (Hunter Kelly)
Date: 03 Dec 1998 14:33:16 -0800
Subject: [Types-sig] Q: What are Meta-classes anyhoo?
In-Reply-To: "Evan Simpson"'s message of "Thu, 3 Dec 1998 14:10:03 -0600"
References: <28da01be1ef8$ea915bb0$2d232cd1@linwin.token.hapenney.com>
Message-ID: <yvt4src97pv.fsf@corona.pixar.com>

I'm gonna throw my hat into the ring on what I've always thought
meta-classes/were/should be.  I'm going to talk about what I
understand Smalltalk's object scheme to be, with the caveat that I
might be wrong or confused.  There is definitely an area where my
understand breaks down, and it is deep into the metaclass thing.

I'm responding to Evan's mail because what I'm gonna talk about
actually covers three or four of the points he mentioned.  Hopefully,
I'll explain them.

So, basic assumptions about Smalltalk  (this taken almost directly
from the Smalltalk-80 book):

_Everything_ is an object.
An object is an instance of a class.

Thusly, "...classes themselves must be represented by instances of a
class.  A class whose instances are themselves classes is called a
_metaclass_."  

Further on, it goes on to say, 
"Whenever a new class is created, a new metaclass is created for it
automatically.  Metaclasses are similar to other classes because they
contain methods used by their instances.  Metaclasses are different
from other classes because they themselves are not instances of
metaclasses.  Instead, they are all instances of a class called
Metaclass."

So, like Evan's #6, metaclasses contain class methods.  In actuality,
in Smalltalk, constuctors are just class methods, ie, methods of
class's metaclass.

From what I understand, this is pretty much what JvR is talking
about (Evan's #5).

I actually like this thing this way.

It takes care of the type vs class debate, in that there would be no
dichotomy. It makes things pretty uniform, ie, any instance gets its
information from it's class.  For normal objects, this is the obvious
thing, for classes, it means it gets its info from it's metaclass.

I think with a little bit of thought you could extend it to define/change how 
an instances attributes are looked up, etc, etc.  I think it could be
extended to all kinds of fancy things.

A friend of mine once mentioned that, in Smalltalk, he could subclass
Metaclass in such a way as to create a very cool transparent debugging 
scheme.

It is definately a good starting point, and worth considering, in my
opinion.  Python  seems to take the best bits and pieces from various
languages, and, in my opinion, this is one (of many) things that
Smalltalk got right.

Hunter


"Evan Simpson" <evan@tokenexchange.com> writes:

> I meant to keep up with the the discussion following my overly-bold
> meta-class post (provoking discussion was my intent, and it has been amply
> fulfilled), but have been ill and cut off from the Internet for two days.
> Now I've read it all, and I'm going to dump mental core all over the place.
> 
> In my response to GvR's ASCII repost, I realized that I did not have a
> decent grasp of what other folks were proposing.  Now I don't even know what
> *I* mean by a meta-class any more.
> 
> Just van Rossum, Don Beaudry, and MA Lemburg all seem to have roughly the
> same idea of what meta-classes would be, and what they would be *for*, but
> this idea isn't made explicit, at least enough for my poor brain to get a
> handle on it.
> 
> John Skaller seems to have a very different take, viewing meta-classes as a
> tool for meta-programming.  This seems to involve the ability to make class
> factories, which would construct complex functions from simple functions by
> much more regular and analytically sound methods than simply compiling a
> string.
> 
> I'm not sure where Tim Peters stands, except that John's approach really
> bothers him <wink>.
> 
> As I said, I'm dumping core here, so here's a list of my thoughts and
> questions in no particular order:
> 
> 1. We really want to do away with types vs. classes, and making types *be*
> classes is the general favorite approach.  Since every object has a type,
> this means that every object will have a class.  Including classes (thus
> meta-class).
> 
> 2. Types are inert lumps, good only for identity comparisons with other
> types.  Classes provide default attributes and methods to their instances.
> Classes *get* default attributes and methods from their base classes, in a
> very powerful and useful way.  What should they get from their meta-class?
> 
> 3. JvR+DB+MAL, if I'm reading them right, have meta-classes defining or
> possibly subverting what it means for an object to be a class or an
> instance.  Am I reading this right?
> 
> 4. For example, MAL has the meta-object "Instance" of an instance (which is
> *not* its class!) control how the instance finds attributes outside its
> dictionary.
> 
> 5. JvR would have the class of the class of the instance take this meta-
> role.  Similarly, the class of the class of a class would define attribute
> search and instance creation for the class <head: boom>
> 
> 6. DB, on the other hand, seems to suggest in one message that a meta-class'
> methods might merely define static methods of its classes, just as class
> methods define static (read "bound") methods of its instances.  I sort of
> took this view initially.
> 
> 7. Since attribute search behavior is the only example I've seen of the
> meta- role's functionality, it's not clear to me how this is superior to
> sub-classing with a mixin.  Mr. Lemburg's __meta__ stuff in particular
> bemuses and befuddles me, since it seems to have little or nothing to do
> with point #1.
> 
> 8. Another point of view (Mr. Skaller's, I think) is that class objects
> build instance objects, albeit in a fairly passive way.  Thus, meta-classes
> should build classes, but in a more active fashion than simply acting as a
> template/default-attribute map (which base classes do anyhow).  I have vague
> visions of filling in class methods by combining sub-function code snippets
> in interesting ways.
> 
> 9. Might this be too static, since changing a meta-class would be like
> changing a class' __init__ rather than like changing its dictionary?  That
> is, the meta-class is only executed to build the class, not dynamically
> searched.
> 
> 10. There is no point #10.
> 
> _______________________________________________
> Types-SIG mailing list
> Types-SIG@python.org
> http://www.python.org/mailman/listinfo/types-sig


From tim_one@email.msn.com  Fri Dec  4 04:36:23 1998
From: tim_one@email.msn.com (Tim Peters)
Date: Thu, 3 Dec 1998 23:36:23 -0500
Subject: [Types-sig] Q: What are Meta-classes anyhoo?
In-Reply-To: <28da01be1ef8$ea915bb0$2d232cd1@linwin.token.hapenney.com>
Message-ID: <000101be1f3f$a5a1d600$aba02299@tim>

[Evan Simpson]
> ...
> I'm not sure where Tim Peters stands,

Like you, Tim has been ill, way too busy, and is having major email problems
too.  I'll let you know where he stands once he's caught up enough that he
thinks he knows himself.

> except that John's approach really bothers him <wink>.

I have more sympathy for John's technical approach than he'll ever know.
His approach to "persuasion" does drive me batty, though -- takes a long
time to digest one of his posts, and in the end trying to extract a concrete
suggestion is like pulling teeth (something I've also had too much
experience with lately <drool>).  Reminds me of this justly famous line from
the 1979 Preface to G. Spencer-Brown's odd little "Laws of Form":

    I did not think then, having given the world the method, that I
    should eventually be required to apply it to the problem myself.

> 10. There is no point #10.

11. Point #10 is no longer operative; however, there is no point #11.

metapoint-to-close-a-metapost-ly y'rs  - tim




From skaller@maxtal.com.au  Fri Dec  4 14:45:02 1998
From: skaller@maxtal.com.au (John Skaller)
Date: Sat, 05 Dec 1998 00:45:02 +1000
Subject: [Types-sig] Q: What are Meta-classes anyhoo?
Message-ID: <1.5.4.32.19981204144502.00f550a8@triode.net.au>

At 14:10 3/12/98 -0600, Evan Simpson wrote:

>8. Another point of view (Mr. Skaller's, I think) is that class objects
>build instance objects, albeit in a fairly passive way.  Thus, meta-classes
>should build classes, but in a more active fashion than simply acting as a
>template/default-attribute map (which base classes do anyhow).  I have vague
>visions of filling in class methods by combining sub-function code snippets
>in interesting ways.

        to give a concrete example from an exitsing language:

        template<class T> class list { ... }

You see that 'list' is a meta-class. It is something
which _constructs_ classes.

You will note that C++ has a serious limitation: there is no
meta-meta-class which constructs meta-classes. This is needed.
For example, to construct a 'product' object you need
something like:

        template<class I> class struct<i> {
                for i in I 'T(i) val(i);'
        }

I have done this: I had to write a C program to generate the
header files for the first n values of i.

        I good Python solution would not have the C++ restriction.
-------------------------------------------------------
John Skaller    email: skaller@maxtal.com.au
		http://www.maxtal.com.au/~skaller
		phone: 61-2-96600850
		snail: 10/1 Toxteth Rd, Glebe NSW 2037, Australia




From skaller@maxtal.com.au  Fri Dec  4 14:45:00 1998
From: skaller@maxtal.com.au (John Skaller)
Date: Sat, 05 Dec 1998 00:45:00 +1000
Subject: [Types-sig] types, functional stuff.
Message-ID: <1.5.4.32.19981204144500.00926634@triode.net.au>

>Now, John Skaller asked for:
>
>>        compose(f1, f2, f3, f4 ..) # chain in series, 1 arg only
>>        product(f1, f2, f3, f4, ..) # apply componentwise (in parallel)
>>        sum(f1, f2, .. ) # select one of
>>        delta(x,i) = (x,x,x,x,x ..) # i times
>
>so, John: there follow implementations for 3 of these in python

        I already have working implementation in interscript's felix
package.

> The language doesn't need them as built-ins.  

        Yes it does. Your implementation is the same as mine,
and it doesn't work for the same reason: we both use delagation.
This binds the result to the _current_ definition of a function,
not the definition at the point of call.

        For actual functions, this is no problem. They're
immutable. But the compose object is not. It's a class instance
whose definition can be changed by changing the args list.
-------------------------------------------------------
John Skaller    email: skaller@maxtal.com.au
		http://www.maxtal.com.au/~skaller
		phone: 61-2-96600850
		snail: 10/1 Toxteth Rd, Glebe NSW 2037, Australia




From evan@tokenexchange.com  Fri Dec  4 16:27:18 1998
From: evan@tokenexchange.com (Evan Simpson)
Date: Fri, 4 Dec 1998 10:27:18 -0600
Subject: [Types-sig] Q: What are Meta-classes anyhoo?
Message-ID: <2add01be1fa2$f6f66b20$2d232cd1@linwin.token.hapenney.com>

>Further on, it goes on to say,
>"Whenever a new class is created, a new metaclass is created for it
>automatically.  Metaclasses are similar to other classes because they
>contain methods used by their instances.  Metaclasses are different
>from other classes because they themselves are not instances of
>metaclasses.  Instead, they are all instances of a class called
>Metaclass."
>
>So, like Evan's #6, metaclasses contain class methods.  In actuality,
>in Smalltalk, constuctors are just class methods, ie, methods of
>class's metaclass.


Eeek.  I like the part about class methods, but the business of
automatically generating a new metaclass behind the scenes for every new
class is a bit odd.  If all metaclasses are instances of Metaclass, and you
can subclass Metaclass (say with MyMetaClass), how do you get an instance of
MyMetaClass to be the metaclass of a new class? Or does Smalltalk let you
implicitly define your metaclass in the class definition somehow?

>I think with a little bit of thought you could extend it to define/change
how
>an instances attributes are looked up, etc, etc.  I think it could be
>extended to all kinds of fancy things.


Now that I think about it, if we have static class methods, and they are not
only bound to the class but to the instance on which they are called as an
argument (None if called on the class), then we could specify that the
various hooks (__getattr__, etc.) must be called *before* the instance dict
is examined if the hook is a static class method.  Then we could have:

... defining metaclass MC...
  def __getattr__(klass, inst, attr):
    if inst is not None:
      # fake up the attribute for the instance inst
    else:
      # fake up the attribute for the class klass
  def astatic(klass, inst, anarg):
    # similar

... later ...
C = MC()
c = C()
# C.x ALWAYS evaluates as MC.__getattr__(C, None, 'x')
# c.x ALWAYS evaluates as MC.__getattr__(C, c, 'x')
# C.astatic(None, 'foo') is MC.astatic(C, None, 'foo')
# C.astatic(c, 'foo') and c.astatic('foo') are MC.astatic(C, c, 'foo')

Of course, this would mean that we would either have to extend the notation
naturally for (meta){2,}classes, or be unable to access more than two
instanciations down.  Also, the interpreter would be forever having to walk
the meta-chain before doing the simplest operations.  Grrr.  Let's put
static class methods into the dictionary of everything they're bound to, and
to heck with dynamic behavior; What sane person wants to mess with an
object's (meta){1,}class at rutime anyway? <0.5 wink>.



From Donald Beaudry <donb@tensilica.com>  Fri Dec  4 18:44:01 1998
From: Donald Beaudry <donb@tensilica.com> (Donald Beaudry)
Date: Fri, 04 Dec 1998 13:44:01 -0500
Subject: [Types-sig] Q: What are Meta-classes anyhoo?
References: <2add01be1fa2$f6f66b20$2d232cd1@linwin.token.hapenney.com>
Message-ID: <199812041844.NAA16403@hq.tensilica.com>

"Evan Simpson" <evan@tokenexchange.com> wrote,
> >Further on, it goes on to say,
> >"Whenever a new class is created, a new metaclass is created for it
> >automatically.  Metaclasses are similar to other classes because they
> >contain methods used by their instances.  Metaclasses are different
> >from other classes because they themselves are not instances of
> >metaclasses.  Instead, they are all instances of a class called
> >Metaclass."
> >
> >So, like Evan's #6, metaclasses contain class methods.  In actuality,
> >in Smalltalk, constuctors are just class methods, ie, methods of
> >class's metaclass.
> 
> Eeek.  I like the part about class methods, but the business of
> automatically generating a new metaclass behind the scenes for every
> new class is a bit odd.

I thought so too, so when I implemented the metaclases for my
objectmodule I didnt bother to make a new metaclass if no changes were
specified... that is, a new metaclass isnt needed if the metaclass
associated with the base class will do.

> If all metaclasses are instances of Metaclass, and you can subclass
> Metaclass (say with MyMetaClass), how do you get an instance of
> MyMetaClass to be the metaclass of a new class?

I am not exactly sure, but..

> Or does Smalltalk let you implicitly define your metaclass in the
> class definition somehow?

Yes.  When defining a class in Smalltalk you must qualify your methods
and members as either class or instance thingies (I forget the exact
keywords needed).  So, when ever you specify a class method, you are
really added a method to the meta-classs's namespace.  When the method
is invoked "self" is the class, not an instance of the class.

So... when you look at the syntax of how classes and metaclass are
specified, it seems pretty reasonable to force the existance of a
metaclass for each class.  This also means, btw, that each metaclass
has only one instance.  It sounds simple and is almost easy to
explain, even if it is somewhat artificial.

For what I thought to be similar reasons, my objectmodule does not
create a metaclass for each class.  Rather, it only creates one when
methods or members of the metaclass are explicitly specified.  In
other words, with my system you dont get a new metaclass unless you
ask for one.  Asking for one in my system is much less subtle than
how you ask for one in Smalltalk.

	--Don




From JOrendorff@ixl.com  Fri Dec  4 23:55:57 1998
From: JOrendorff@ixl.com (JOrendorff@ixl.com)
Date: Fri, 04 Dec 1998 15:55:57 -0800
Subject: [Types-sig] types, functional stuff.
References: <1.5.4.32.19981204144500.00926634@triode.net.au>
Message-ID: <3668768D.C0CAD922@ixl.com>

> > The language doesn't need them as built-ins.
> 
>         Yes it does. Your implementation is the same as mine,
> and it doesn't work for the same reason: we both use delagation.
> This binds the result to the _current_ definition of a function,
> not the definition at the point of call.
> 
>         For actual functions, this is no problem. They're
> immutable. But the compose object is not. It's a class instance
> whose definition can be changed by changing the args list.

So don't change the args list.

-- 
Jason


From just@letterror.com  Fri Dec  4 22:38:28 1998
From: just@letterror.com (Just van Rossum)
Date: Fri, 4 Dec 1998 23:38:28 +0100
Subject: [Types-sig] Why have two hierarchies? (*EUREKA!*)
In-Reply-To: <3.0.3.32.19981203222234.0093ccc0@pop.xs4all.nl>
References: <28da01be1ef8$ea915bb0$2d232cd1@linwin.token.hapenney.com>
Message-ID: <l03102802b28d86ad4077@[193.78.237.156]>

(For those of you with a short attention span, you can skip right through
to the section marked with ******)

At 10:22 PM +0100 12/3/98, Niels Ferguson wrote:
>I have been trying to follow the meta-class discussion (with varying
>success), so it is quite possible that my comments are due to my
>limited understanding of the subject.

I may have posted a few times on the subject, but trust me, most of the
time I feel just like that. (I gave up on trying not to embarrass Guido
with my stupidity a long time ago: I might as well not post anything ;-)

>I find the idea of having a meta-class hierarchy and a separate class
>derivation hierarchy confusing.

It is.

>My question is whether you cannot merge the
>two into a single framework.

After I first read your mail I wrote a little reply, which explained why
this is neccesary, and before I got to the end of the paragraph it occurred
to me that it is horse-shite. So: thankyou thankyou thankyou. I feel much
better now.

The complexity of all meta proposals I have seen so far (now, _maybe_ I
missed a few, and I definitely don't know Smalltalk, so there's your
disclaimer) -- including mine *and* Guido's -- are mindboggling. I mean, if
my head explodes every half hour about hing I make up myself, there's
something seriously wrong.

So I wrote up Yet Another Object Protocol Proposal. I tried to avoid the
M-word: It's true, the word itself turns your mind into an infinite
recursion. Still, I won't talk about Turtles either.

Btw, my new proposal nullifies many points I tried to make in my previous
postings, including my very own meta hook!


******

A proposal for a flat object hierarchy (a.k.a. "The Earth is Flat")


Executive summary

                   An instance of a class
                   is the _same_ thing as
                   a subclass of a class.


"Reserved" attributes

    - Every object has a __bases__ attribute, which contains
      a sequence of arbitrary length containing other objects.
      Possibly classes.

    - Every object has a __namespace__ attribute, which is a dict.
      This is much the same as the current __dict__ but to avoid
      confusion I picked a new name. It also makes implementation
      possible with Today's Python.

    - Every object has a __name__ attribute.

These three attributes are reserved. Access the these is handled by the
underlying implementation and cannot be trapped by users. In an actual
implementation in, say, C++ __namespace__ and __name__ could be optimized
out for very simple objects (numbers, strings), but __bases__ should always
be there.

Time for a random discision. Since I already got rid of the fundamental
difference between classes and instances, I needed a way to differentiate
them (!). This is because of the bound/unbound method issue. I chose to use
the __name__ attribute as a flag: if it is None, we're dealing with in
instance. If it's not None we're dealing with a class.


Attribute lookup

    - If the object's __bases__ sequence is empty (ie. no bases),
      fall back to the simple-most getattr scheme: see if
      object.__namespace__ contains the attribute.

    otherwise:

    - Loop through object.__bases__ and ask for a "__getattr__"
      attribute.

    - If one is found, call it, it'll return the attribute or
      raises AttributeError

    - Check whether we are an instance or a class (ie. check at the
      __name__ attribute), make methods bound or unbound accordingly.

Methods

Methods in my scheme are identical to instance methods as we know them in
Python today:

    - The "im_func" attribute, which is the actual function object

    - The "im_class" attribute, which is the class in which the method
      was found.

    - The "im_self" attribute, which is the instance to which the
      method is bound (if any, so may be None).

I defined a new method for the method object, mainly for internal purposes:

    - The "bind" method. It takes an instance as argument, and
      replaces the "im_self" attribute with it.

That's not illegal: that's _cool_. The attribute lookup scheme uses itself
recursively to get inherited attibutes. Way at the end of the scheme we
test whether we are dealing with a class or an instance and either make
methods bound or unbound.

Since these methods will always be constructed on the fly there is no
danger in binding it more than once: it will only get _used_ after the
attribute lookup is completed.

(Implementation note: if we'd like to do method caching, which will quite
likely be an enormous performance boost, one could implement a method lock.
method.bind() will raise an exception when the method is "locked".)


Consequences

The main differences with the current Python object model are:

   1) A "custom" __getattr__ will by definition _always_ be invoked,
      and not only after "normal" lookup failed. This is because
      this _is_ the normal lookup procedure.

   2) No types, everything is handled through object.__bases__.

   3) No __class__ attribute: everything is done through
      object.__bases__.

The difference with other proposals seems to be:

   1) There are not _really_ meta classes involved. In this scheme,
      a meta class is conceptually identical to a base class. Since
      any object can have multiple base classes, this conceptually
      means objects can have multiple meta classes. What this is
      useful for, I will leave up to others.

I agree with Gordon that the current meta class stuff seems only
useful for fancy getattr tricks. Since point (1) makes that possible
(as a consequence of concept of this object model, not as a cheap
trick!), we can _really_ do away with the concept of meta classes.
Poof! Nobody needs to understand. There is nothing to understand!
If someone with a Smalltalk background steps up and asks: how do
I write a meta class in Python, the answer will be: write a class,
then subclass it. "But that's not the same!" the may sputter. Well,
it is now.

It's so simple I can hardly believe anybody in this group will buy it.
Maybe it could actually be useful! Must destroy...



Appendix A

And now for the hottest part:
   *** This Actually Works in Plain Vanilla Python 1.5! ***

On this page you'll find an implementation:

    http://starship.skyport.net/~just/meta/Flat.py

Exhaustively commented. Sorry, no energy left for demos ("batteries not
included"). Look at the test code at the bottom for simple examples of how
compatible this is with current Python semantics.
For those of you who are lazy: it's very compatible.



Just




From mal@lemburg.com  Fri Dec  4 23:21:18 1998
From: mal@lemburg.com (M.-A. Lemburg)
Date: Sat, 05 Dec 1998 00:21:18 +0100
Subject: [Types-sig] Why have two hierarchies? (*EUREKA!*)
References: <28da01be1ef8$ea915bb0$2d232cd1@linwin.token.hapenney.com> <l03102802b28d86ad4077@[193.78.237.156]>
Message-ID: <36686E6E.2AEB32D7@lemburg.com>

Just van Rossum wrote:
> 
> And now for the hottest part:
>    *** This Actually Works in Plain Vanilla Python 1.5! ***
> 
> On this page you'll find an implementation:
> 
>     http://starship.skyport.net/~just/meta/Flat.py
> 
> Exhaustively commented. Sorry, no energy left for demos ("batteries not
> included"). Look at the test code at the bottom for simple examples of how
> compatible this is with current Python semantics.
> For those of you who are lazy: it's very compatible.

FYI, If you want to see the above script in color, go for:

http://starship.skyport.net/~lemburg/py2html.cgi/~just/meta/Flat.py

or more general:

http://starship.skyport.net/~lemburg/py2html.cgi/http://starship.skyport.net/~just/meta/Flat.py

Note that this WWW colorizer works for all Python scripts available
on the web: just add the complete URL behind '.../py2html.cgi/' and
you get the script in color. For scripts lying on starship you
can strip the 'http://starship.skyport.net/' part.

-- 
Marc-Andre Lemburg                               Y2000: 392 days left
---------------------------------------------------------------------
          : Python Pages >>> http://starship.skyport.net/~lemburg/  :
           ---------------------------------------------------------



From just@letterror.com  Sat Dec  5 13:42:54 1998
From: just@letterror.com (Just van Rossum)
Date: Sat, 5 Dec 1998 14:42:54 +0100
Subject: [Types-sig] Why have two hierarchies? (*EUREKA!*)
In-Reply-To: <l03102802b28d86ad4077@[193.78.237.156]>
References: <3.0.3.32.19981203222234.0093ccc0@pop.xs4all.nl>
 <28da01be1ef8$ea915bb0$2d232cd1@linwin.token.hapenney.com>
Message-ID: <l03102800b28ee586b8eb@[193.78.237.169]>

Here's one additional remark that I forgot to put in the article.

The meta classes issue comes from this question:

    If an object has a class (a __class__ attribute), the what is
    the class of the object's class?

My (current ;-) solution to this infinite recursion can be summarized thusly:

Today, instances have a __class__, and classes have base classes. Instead
of adding a __class__ attribute to classes, we should get rid of the
__class__ attribute of instances and give them __bases__ instead. So
instead of _adding_ a layer of complexity (meta classes) I _remove_ one.

Python today:

    instance.__class__.__bases__

I suggest:

    instance.__bases__

Ok, enough already. Let the hangman come!

Just





From Donald Beaudry <donb@tensilica.com>  Sat Dec  5 17:45:15 1998
From: Donald Beaudry <donb@tensilica.com> (Donald Beaudry)
Date: Sat, 05 Dec 1998 12:45:15 -0500
Subject: [Types-sig] Why have two hierarchies? (*EUREKA!*)
References: <3.0.3.32.19981203222234.0093ccc0@pop.xs4all.nl> <28da01be1ef8$ea915bb0$2d232cd1@linwin.token.hapenney.com> <l03102800b28ee586b8eb@[193.78.237.169]>
Message-ID: <199812051745.MAA31791@hq.tensilica.com>

Just van Rossum <just@letterror.com> wrote,
> Python today:
> 
>     instance.__class__.__bases__
> 
> I suggest:
> 
>     instance.__bases__
> 
> Ok, enough already. Let the hangman come!

But that's too simple to be any fun ;) Besides, now you have to answer
the question of what object is used to represent the bases collection?
Is it a tuple of objects, or is it just an object?  ...and what about
the objects in the collection, what can we expect of them.  What
attributes and methods do they have?  What attributes and methods does
the collection have?  Where do we encode the attribute look-up policy
for our instances?

You can change the names all you want, but simplifying this mess isnt
going to be that simple :)

	--Don


From evan@tokenexchange.com  Sat Dec  5 18:33:35 1998
From: evan@tokenexchange.com (Evan Simpson)
Date: Sat, 5 Dec 1998 12:33:35 -0600
Subject: [Types-sig] Why have two hierarchies? (*EUREKA!*)
Message-ID: <2c7701be207d$c5a68ce0$2d232cd1@linwin.token.hapenney.com>

<Applaud!><Whistle!><Cheer!><WooHoo!>

You've had a genuine Aha! moment, and what's more you've shared it
successfully.  I thank you.

Despite your excellent exposition, I still feel the need to sum up, if only
to confirm my understanding.

*** summary, with one question ***

1. Objects is objects.  Period.  They can store stuff in their namespace.

2. Objects are based on other objects.  This forms an inheritance/delegation
DAG exactly like current class DAGs.

3. If you give an object a name, you've made a class.  This allows you to
search the inheritance DAG by name.

4. Simple attribute operations on a baseless object act on its namespace.

5. A based object checks to see if its base objects want to control
attribute operations.  If not, it falls back on #4, then asks the base
objects for attributes not found in its namespace.

6. When a function 'f' is an attribute of a named object, the object's kids
see 'f' as a method.  Named kids see 'f' unbound, unnamed kids see it bound
to themselves.  QUESTION: How do bound methods behave? Do they require an
unnamed object (an 'instance')?

7. (Not sure on this one) An object can bind a method to itself before
handing it to its kids, thus allowing the elusive static class method to be
realized.

8. The type/class of an object is its base tuple, or None for baseless
objects.

*** consequences and opportunities ***

9. "class A(B, C):" constructs an object with bases (B, C), names it "A",
and binds it to the local namespace as "A".

10. "a=A()" constructs an object with base (A,) and no name, and binds it to
the local namespace as "a".

What about:
11. "class A2(a):" constructs an object as in #9, but its base is the
unnamed object from #10.
Do we allow this, or require bases to be named? If we allow it, do we allow
"b = a()"?  How does this mix with __call__?

12. "a = class (A):" or "a = object (A):" could be a synonym for #2, but
allow a code suite to execute in the context of 'a's namespace.

13. "a = class:" or "a = object:" could perform the same function for
baseless objects.
We would now have a simple way to spell nested structures:

a = object:
  a1 = 1
  def null(): pass
  a2 = object:
    b = "hi!"
#a.a2.b == "hi!"
#a.null is a function, not a method!

14. "5.__base__ is type(5) is Integer", "5" is nameless, and "
type(5).__name__=='Integer' ".

15. The holy grail "class myInt(Integer):" is attainable.

*** last minute thought ***

I am bothered by the need to search the inheritance DAG for attribute
operators every time you use one.  Caching and other possible optimizations
don't make me feel any better - I can't say exactly why.

Suppose objects could have a __birth_hook__ method, called every time a new
descendent is created.  Unlike __init__, this would be called on named and
unnamed objects alike, and before __init__ for unnamed objects.  Objects
which redefine __init__ have to cooperate with ancestral __init__s by
calling them explicitly, but __birth_hook__ is forced on them.  It would be,
among other things, a chance to install __*attr__ hooks into the namespace
of the new object.  "Static class methods" could be installed in subclasses
(kids with names).  We could keep the current namespace-then-delegate model,
but still allow hookers ;-)



From just@letterror.com  Sat Dec  5 19:00:21 1998
From: just@letterror.com (Just van Rossum)
Date: Sat, 5 Dec 1998 20:00:21 +0100
Subject: [Types-sig] Why have two hierarchies? (*EUREKA!*)
In-Reply-To: <199812051745.MAA31791@hq.tensilica.com>
References: <3.0.3.32.19981203222234.0093ccc0@pop.xs4all.nl>
 <28da01be1ef8$ea915bb0$2d232cd1@linwin.token.hapenney.com>
 <l03102800b28ee586b8eb@[193.78.237.169]>
Message-ID: <l03102805b28f30f07b30@[193.78.237.160]>

Donald Beaudry wrote:
>But that's too simple to be any fun ;)

I *knew* it! ;-)

>Besides, now you have to answer
>the question of what object is used to represent the bases collection?
>Is it a tuple of objects, or is it just an object?

As far as I'm concerned any sequence object will do. In other words: any
object that implements __getitem__.

>...and what about
>the objects in the collection, what can we expect of them.

- that they have a __bases__ attribute
- that they have a __namespace__ attribute (without one, inheriting
  makes no sense)
- that you can fetch attributes from them.

Nothing more I guess.

>What
>attributes and methods do they have?

I'd say a __getattr__ method.

>What attributes and methods does
>the collection have?

I answered that before: __getitem__...

>Where do we encode the attribute look-up policy
>for our instances?

In a custom __getattr__. That may sound primitive, but that's all there is
to it (as far as I can see). If you define a __getattr__ it will get called
for *all* attribute access: if you want to pretend you're inheriting of
something else than what __bases__ says, go right ahead! Only your
__getattr__ method must be on the __bases__ path.

>You can change the names all you want, but simplifying this mess isnt
>going to be that simple :)

What's the problem? ;-) (Seriously: I *may* just not get it: please
elaborate...)

Just




From just@letterror.com  Sat Dec  5 19:47:18 1998
From: just@letterror.com (Just van Rossum)
Date: Sat, 5 Dec 1998 20:47:18 +0100
Subject: [Types-sig] Why have two hierarchies? (*EUREKA!*)
In-Reply-To: <2c7701be207d$c5a68ce0$2d232cd1@linwin.token.hapenney.com>
Message-ID: <l03102806b28f33560bca@[193.78.237.160]>

Evan Simpson wrote:
><Applaud!><Whistle!><Cheer!><WooHoo!>

<bow>

(-- sounds of grumbling demi-gods in the back of the room --)

Cool summary, mostly correct. A couple of comments and a question in return.

>2. Objects are based on other objects.  This forms an inheritance/delegation
>DAG exactly like current class DAGs.

What is a DAG?

>3. If you give an object a name, you've made a class.

That *could* be a possibility. So far, it makes a lot of sense to me, but
this detail is not very important: as long as there is _some_ way to
distinguish a class form an instance.

>This allows you to search the inheritance DAG by name.

Sorry, I don't follow. What is a DAG?

>6. When a function 'f' is an attribute of a named object, the object's kids
>see 'f' as a method.  Named kids see 'f' unbound, unnamed kids see it bound
>to themselves.  QUESTION: How do bound methods behave? Do they require an
>unnamed object (an 'instance')?

Not as far as I'm concerned: a bound method will probably automatically
have an im_self attribute of the right class, but if you're calling an
unbound method I strongly feel it's the responsibility of the caller to
pass an object (*whatever* object) that the unbound method can handle.

>7. (Not sure on this one) An object can bind a method to itself before
>handing it to its kids, thus allowing the elusive static class method to be
>realized.

Not sure about this one myself. The easy way is to keep doing what we've
always done:
    MyClass.foo(MyClass, ...)

>What about:
>11. "class A2(a):" constructs an object as in #9, but its base is the
>unnamed object from #10.
>Do we allow this, or require bases to be named? If we allow it, do we allow
>"b = a()"?

That maybe a fuzzy part of my scheme indeed. I haven't fully thought this
part through yet.

>How does this mix with __call__?

My demo implementation is broken in this respect. After I posted the stuff
I thought of the obvious solution. Hm, solution is the wrong word: I just
implemented existing Python's call semantics wrongly. It _should_ work like
this:

    - if the object we're calling is an instance, search for a
      __call__ method
    - if the object we're calling is a class, do one of these (don't know
      what's better, possibly the latter):
        a) construct a new class with (name, (self,), {}) as arguments,
           search for an __init__ method, call it if found. (just like
Python Classic)
      or
        b) search for a __instantiate__ method, let it handle (a)


>12. "a = class (A):" or "a = object (A):" could be a synonym for #2, but
>allow a code suite to execute in the context of 'a's namespace.

I don't quite follow...

>13. "a = class:" or "a = object:" could perform the same function for
>baseless objects.
>We would now have a simple way to spell nested structures:
>
>a = object:
>  a1 = 1
>  def null(): pass
>  a2 = object:
>    b = "hi!"
>#a.a2.b == "hi!"
>#a.null is a function, not a method!

Hmmm, not bad... But it is the same as:

class a:
    a1 = 1
    a1 = 1
    def null(): pass
    a2 = object:
       b = "hi!"

a.__name__ = None

;-)

>14. "5.__base__ is type(5) is Integer", "5" is nameless, and "
>type(5).__name__=='Integer' ".

I don't know what you mean by type... But yes, something in __bases__ makes
it an integer.

>15. The holy grail "class myInt(Integer):" is attainable.

Phew.

>*** last minute thought ***
>
>I am bothered by the need to search the inheritance DAG for attribute
>operators every time you use one.  Caching and other possible optimizations
>don't make me feel any better - I can't say exactly why.

I have similar feelings. Right now, my helper method __getattr_from_bases__
uses __getattr__ on each of the bases. Maybe it should just recursively
search their __namespaces__. If you want to override that behaviour: just
define a custom __getattr__ which does what you want.

>Suppose objects could have a __birth_hook__ method, called every time a new
>descendent is created.

Right, this is exactly what I meant with __instantiate__ above.

>Unlike __init__, this would be called on named and
>unnamed objects alike, and before __init__ for unnamed objects.

Erm, confusion: are you creating an unnamed object from a named base or a
named or unnamed object from an unnamed object???

>Objects
>which redefine __init__ have to cooperate with ancestral __init__s by
>calling them explicitly, but __birth_hook__ is forced on them.  It would be,
>among other things, a chance to install __*attr__ hooks into the namespace
>of the new object.  "Static class methods" could be installed in subclasses
>(kids with names).  We could keep the current namespace-then-delegate model,
>but still allow hookers ;-)

At the conference hotel in Houston, there was a bar called "Hookers Nook"...

Just





From evan@tokenexchange.com  Sat Dec  5 22:31:38 1998
From: evan@tokenexchange.com (Evan Simpson)
Date: Sat, 5 Dec 1998 16:31:38 -0600
Subject: [Types-sig] Why have two hierarchies? (*EUREKA!*)
Message-ID: <2c9801be209f$0730a530$2d232cd1@linwin.token.hapenney.com>

One of these days I'll remember to make my posts as brief as possible *but
no briefer* :-)

>>2. Objects are based on other objects.  This forms an
inheritance/delegation
>>DAG exactly like current class DAGs.
>
>What is a DAG?

Directed Acyclic Graph, or what you get with a multiple inheritance
hierarchy.  Sorry, I didn't realize I was being obscure.

>>3. If you give an object a name, you've made a class.
>
>That *could* be a possibility. So far, it makes a lot of sense to me, but
>this detail is not very important: as long as there is _some_ way to
>distinguish a class form an instance.

It makes a lot of sense to me too, and I really like the idea that classes
have a name independently of how you got hold of  them, but instances are
anonymous except for their parentage.

>>This allows you to search the inheritance DAG by name.
>
>Sorry, I don't follow. What is a DAG?

I was being too concise again.  I just meant that you can search the
object's bases, and the bases' bases, etc for a class with a particular
name, without actually having a reference to that class.

[snippage] if you're calling an
>unbound method I strongly feel it's the responsibility of the caller to
>pass an object (*whatever* object) that the unbound method can handle.

I have mixed feelings here.  Eliminating the concept of unbound methods in
favor of raw functions makes the semantics more powerful and general, at the
cost of losing the automatic catching of "I forgot the self argument"
errors.

>>7. (Not sure on this one) An object can bind a method to itself before
>>handing it to its kids, thus allowing the elusive static class method to
be
>>realized.
>
>Not sure about this one myself. The easy way is to keep doing what we've
>always done:
>    MyClass.foo(MyClass, ...)

Since when have we been able to do this?  It doesn't work in the general
case, anyway, since a method may wish to operate on the state of classes
which are one or more subclasses removed from the one in which it was
defined, while allowing you to invoke it on any further subclass or
instance.  Invoke such a method on an instance, even, and it has no reliable
way of finding the parent class it's supposed to be tweaking.

>>12. "a = class (A):" or "a = object (A):" could be a synonym for #2, but
>>allow a code suite to execute in the context of 'a's namespace.
>
>I don't quite follow...

Oops.  Obscure *and* I forgot I re-numbered my points.
I meant that instead of:

a=A()
a.foo = 1
def bar(): pass
a.bar = bar

we could write:

object a(A): #or a = class (A):
  foo = 1
  def bar(): pass

Heck, you could even give instances docstrings in the usual way, if you
wanted.

[example of spelling an anonymous object]

>Hmmm, not bad... But it is the same as:
>
>class a:
>    a1 = 1
>    a1 = 1
>    def null(): pass
>    a2 = object:
>       b = "hi!"
>
>a.__name__ = None
>
>;-)

Hmf.  Only if you want to drive the newbies mad.  If anonymous objects
proved to be popular (as quick&easy containers, say), I'd say there would be
a *strong* case for an "object" keyword to keep the intent explicit.  Do we
*really* want to let people mutate __name__ without being slapped with a
chicken? <whimper>

>>Suppose objects could have a __birth_hook__ method, called every time a
new
>>descendent is created.
>
>Right, this is exactly what I meant with __instantiate__ above.

Wait, I read you as saying that __instantiate__ *or* __init__ would be
called, but not both.  If you meant that they get called in sequence, I
agree.

>>Unlike __init__, this would be called on named and
>>unnamed objects alike, and before __init__ for unnamed objects.
>
>Erm, confusion: are you creating an unnamed object from a named base or a
>named or unnamed object from an unnamed object???

I mean that *everything* descending from a class, whether it be an instance,
a subclass, or an instance of a subclass would get "birthed".  Only the
instances get __init__'ed, just as now.  Under your scheme, after all, the
only difference between subclassing and instantiation is whether the result
is a class (has a name).  The birth hook lets you handle new objects before
anyone else gets to see them.


I-suppose-you-could-call-it-a-midwife-ly^H^H^H^H^H

def Evan(Simpson):
  while self is not in PSA:
    avoid("yr's style signature")
    self.send(self, "Remember to sign up, already")




From just@letterror.com  Sat Dec  5 23:52:13 1998
From: just@letterror.com (Just van Rossum)
Date: Sun, 6 Dec 1998 00:52:13 +0100
Subject: [Types-sig] Why have two hierarchies? (*EUREKA!*)
In-Reply-To: <2c9801be209f$0730a530$2d232cd1@linwin.token.hapenney.com>
Message-ID: <l03102809b28f72d70122@[193.78.237.160]>

I wondered:
>>What is a DAG?

Evan replied:
>Directed Acyclic Graph,

Yeah right, like that would help a poor graphic designer <wink>.

>>Not sure about this one myself. The easy way is to keep doing what we've
>>always done:
>>    MyClass.foo(MyClass, ...)
>
>Since when have we been able to do this?

Oops, sorry. Freudian slip of the keyboard. I meant something like "The
easy way would be to do it like we call an unbound method, but with the
class instead of self." (insert above example)

>we could write:
>
>object a(A): #or a = class (A):
>  foo = 1
>  def bar(): pass

Ok, I see. Time for a tough one: if we fetch a function from an unnamed
object out of its own namespace, should we bind it so self? Might me
probematic if you want to store raw functions in self.__namespace__.

>>Right, this is exactly what I meant with __instantiate__ above.
>
>Wait, I read you as saying that __instantiate__ *or* __init__ would be
>called, but not both.  If you meant that they get called in sequence, I
>agree.

Hm, I just realize I misread you. I meant:

- If you call a named object (ie. a class), that object will be searched
for an __instantiate__ method. If found, call that. It should handle the
whole instantiation and return an object. Maybe even something else... If
not found, default instantiation occurs.
- If you call an anonymous object (ie. an instance), ask it for a __call__
method. If not found: raise an exception.

Whereas you meant something entirely different:

>a subclass, or an instance of a subclass would get "birthed".  Only the
>instances get __init__'ed, just as now.  Under your scheme, after all, the
>only difference between subclassing and instantiation is whether the result
>is a class (has a name).  The birth hook lets you handle new objects before
>anyone else gets to see them.

Right. Good point. Will think about it.

Just




From skaller@maxtal.com.au  Sun Dec  6 00:36:14 1998
From: skaller@maxtal.com.au (John Skaller)
Date: Sun, 06 Dec 1998 10:36:14 +1000
Subject: [Types-sig] Why have two hierarchies? (*EUREKA!*)
Message-ID: <1.5.4.32.19981206003614.00f86208@triode.net.au>

At 14:42 5/12/98 +0100, Just van Rossum wrote:
>Here's one additional remark that I forgot to put in the article.
>
>The meta classes issue comes from this question:
>
>    If an object has a class (a __class__ attribute), the what is
>    the class of the object's class?
>
>My (current ;-) solution to this infinite recursion can be summarized thusly:
>
>Today, instances have a __class__, and classes have base classes. Instead
>of adding a __class__ attribute to classes, we should get rid of the
>__class__ attribute of instances and give them __bases__ instead. So
>instead of _adding_ a layer of complexity (meta classes) I _remove_ one.
>
>Python today:
>
>    instance.__class__.__bases__
>
>I suggest:
>
>    instance.__bases__
>
>Ok, enough already. Let the hangman come!

        This is a very powerful argument.
It seems to me that additional abstaction can only
come from simplifiction. A 'just' justification :-)

        Also, such a simplification, to be truly
worthy, must provide _unexpected_ new functionality.

        Now .. doesn't this scheme look a bit like
supporting delegation ..??

-------------------------------------------------------
John Skaller    email: skaller@maxtal.com.au
		http://www.maxtal.com.au/~skaller
		phone: 61-2-96600850
		snail: 10/1 Toxteth Rd, Glebe NSW 2037, Australia




From just@letterror.com  Sun Dec  6 03:27:43 1998
From: just@letterror.com (Just van Rossum)
Date: Sun, 6 Dec 1998 04:27:43 +0100
Subject: [Types-sig] Why have two hierarchies? (*EUREKA!*)
In-Reply-To: <1.5.4.32.19981206003614.00f86208@triode.net.au>
Message-ID: <l0310280cb28fa8c9b63a@[193.78.237.160]>

John Skaller wrote:
>        This is a very powerful argument.
>It seems to me that additional abstaction can only
>come from simplifiction. A 'just' justification :-)
>
>        Also, such a simplification, to be truly
>worthy, must provide _unexpected_ new functionality.

True. I'm working on a new extended version of the article and a new demo.
This will feature (...drum roll...) class methods. Hmm, maybe that's not
unexpected...

>        Now .. doesn't this scheme look a bit like
>supporting delegation ..??

Could a kind soul please explain to me in a couple of sentences what
delegation *is*? Thanks. "Design Patterns" is patiently waiting for me
beside my bed, but I'd rather have someone just tell me... Yeh, I'm 100%
unqualified to talk about these subjects. It's more fun that way...

Just




From tim_one@email.msn.com  Sun Dec  6 09:02:37 1998
From: tim_one@email.msn.com (Tim Peters)
Date: Sun, 6 Dec 1998 04:02:37 -0500
Subject: [Types-sig] Why have two hierarchies? (*EUREKA!*)
In-Reply-To: <l03102809b28f72d70122@[193.78.237.160]>
Message-ID: <000001be20f7$2bf46340$6b9e2299@tim>

[Just van Rossum]
> What is a DAG?

[Evan Simpson]
> Directed Acyclic Graph,

[Just]
> Yeah right, like that would help a poor graphic designer <wink>.

The problem for you may be that DAG is a Swedish name, as in Dag Hjalmar
Agné Carl Hammarskjöld, former secretary-general of the United Nations.
Which aren't so united after all, so they take turns blaming each other.
Like everyone blames Donald for the metaclass hook, but Donald doesn't blame
anyone back.

That's a graph!  We're the nodes, and our pointing fingers of shame are
arrows connecting the nodes.  Since our fingers have nails, they have
distinguished ends, which is good else we'd never be able to tell who we
were pointing *at*.  And that makes it a directed graph.  Without a
direction to the blame, all of Usenet would collapse!  Which would make life
simpler, so people would rather work with undirected graphs when they can.
Here we can't, though -- someone must pay.

So, in technical language, a directed graph is a bunch of nannies wagging
their fingers at Donald and going "tsk, tsk".  So long as Donald never
points back, a small flying ant following the trail of blame will eventually
come to a place it can sleep, most likely near Donald's nose.  Much as Dag
Hammarskjöld now rests peacefully in death, to which node all life choices
eventually lead in their own directed-graph way.

But if Guido blames Donald and Donald blames Guido right back, that poor ant
will follow those fingers back & forth & back & forth, getting as tired as
any poor reader trying to get thru Dag Hammarskjöld's "Night Is Drawing
Nigh" -- which also never ends.

So if an immortal small flying ant never runs out of pointing fingers to
follow (ant brains aren't big enough to remember which fingers they've
already followed), that's called a cycle.  Mostly because even if the ant
*had* a motorcycle with plenty of gas and euler, it *still* wouldn't find an
end to the blame.  But if the blame always comes to an end, no matter which
pointing finger we start at or which path we follow, then the graph is
called acyclic.  Meaning you always arrive at a dead end even without a
motorcycle.  Things get a bit more complicated if the graph itself is
infinite, but that's an irrelevant distraction so good thing I didn't
mention it.

Now don't go confusing acyclic with acyclovir!  Acyclovir is a drug used
mainly to treat genital herpes infections, which, although surely related to
the type/class split, and despite that they too can be discreetly ignored on
a first date, are much easier to cure.

All right.  Now you're thinking "OK, Uncle Timmy!  I apologize for bringing
venereal diseases into this.  But doesn't acyclic mean 'a cycle'?".  No!  It
actually means *without* a cycle (let alone an ic).  Let's study this
illuminating quote from Dag Hammarskjöld:  "Life yields only to the
conqueror.  Never accept what can be gained by giving in.  You will be
living off stolen goods, and your muscles will atrophy."  See?  He didn't
mean your muscles would get a trophy, he meant they would become so weak &
flabby they would not get a trophy.  It's a quirk of English that even a
Swede once mastered, so a gawky Dutchman's younger psychotic brother should
be able to master it too.  Not to be confused with an agawky Dutchman, which
Just is, meaning not the opposite of gawky but rather without gawkiness
(BTW, Guido isn't gawky at all!  But let's pretend he is, in the interests
of effective pedagogy.).

We haven't even mentioned trolls or butter knives yet, let alone their
connections to saunas and lutefisk.  But it may suffice to leave it at this:
all DAG means in this context is that if you follow the pointing finger of
class inheritance, then no matter which class you (or a small immortal
flying ant -- your choice) start at, or which path you follow, you'll never
come back to the class you started at.  Or, IOW, you'll always reach a dead
end.  Or, IOW, no class is its own anscestor or its own descendant.  Or,
IOW, if X and Y are distinct classes, it's never the case that issubclass(X,
Y) and issubclass(Y, X) are both true.

Now about "delegation", that just means getting someone else to do your work
for you, but in an honest way, not in a way that would make anyone take away
your muscles' trophies.  Like Sweden sent a delegation to the United Nations
to do their collective inter-nation-al negotiating for them.  Or maybe it
was just to get that preachy Dag Hammarskjöld out of their country, but the
end result is the same either way:  they sat on their butts while the
delegation did all the work.

Dag's all, folks!

fatigue-dulls-the-pain-but-awakes-enticing-thoughts-of-death-ly y'rs  - dag




From skaller@maxtal.com.au  Sun Dec  6 11:31:26 1998
From: skaller@maxtal.com.au (John Skaller)
Date: Sun, 06 Dec 1998 21:31:26 +1000
Subject: [Types-sig] Why have two hierarchies? (*EUREKA!*)
Message-ID: <1.5.4.32.19981206113126.00f6a14c@triode.net.au>

At 04:27 6/12/98 +0100, Just van Rossum wrote:
>John Skaller wrote:
>
>>        Now .. doesn't this scheme look a bit like
>>supporting delegation ..??
>
>Could a kind soul please explain to me in a couple of sentences what
>delegation *is*? 

        It's what managers do: get someone else to do the work.
Here's an example:

        class worker:
                def typestuff(self,x): print x

        class boss:
                def __init__(self, aworker):
                        self.aworker = aworker
                def typestuff(self,x):
                        self.aworker.typestuff(x)

Basically, delegation is a more powerful version of inheritance,
in which methods are forwarded to other instances.

If Python had delegation, it would be much easier to 
implement say, a dictionary emulation, which used
an actual dictionary, but changed one or two of the methods.

At present, you'd have to forward every dictionary method call
with a wrapper (forwarding) function. This is a pain.
With 'automatic' delegation, the lookup system will
do it for you. "If you cannot find method f in my
method set, get one from the workers method set and use that".

Delegation is also what 'proxy' objects often do.
The technique is often prefered to inheritance in OO
and non-OO systems because it is more powerful:
one can dynamically change a worker object to another.
And share workers amoung bosses.

The extra power is also the main disadvantage of
delegation, especially in statically typed systems
where the static typing provides guarrantees
which are harder to enforce with delegation.

-------------------------------------------------------
John Skaller    email: skaller@maxtal.com.au
		http://www.maxtal.com.au/~skaller
		phone: 61-2-96600850
		snail: 10/1 Toxteth Rd, Glebe NSW 2037, Australia




From fredrik@pythonware.com  Sun Dec  6 13:36:25 1998
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Sun, 6 Dec 1998 14:36:25 +0100
Subject: [Types-sig] Why have two hierarchies? (*EUREKA!*)
Message-ID: <00e401be2120$0480e9e0$f29b12c2@pythonware.com>

>Could a kind soul please explain to me in a couple of sentences what
>delegation *is*? Thanks. "Design Patterns" is patiently waiting for me
>beside my bed, but I'd rather have someone just tell me... Yeh, I'm 100%
>unqualified to talk about these subjects. It's more fun that way...

 class MyClass:
    def __init__(self, someoneelse):
        self.someoneelse = someoneelse
    def mymethod(self, argument):
        # explicitly delegate mymethod
        return self.someoneelse.mymethod(argument)

or, in a more flexible fashion:

class MyClass:
    def __init__(self, someoneelse):
        self.someoneelse = someoneelse
    def __getattr__(self, attribute):
        # delegate everything not handled by this instance
        return getattr(self.someoneelse, attribute)

I'm using it all the time...

Cheers /F
fredrik@pythonware.com
http://www.pythonware.com




From vorpal@xs4all.nl  Sun Dec  6 20:35:14 1998
From: vorpal@xs4all.nl (Niels Ferguson)
Date: Sun, 06 Dec 1998 21:35:14 +0100
Subject: [Types-sig] Why have two hierarchies? (*EUREKA!*)
In-Reply-To: <l03102802b28d86ad4077@[193.78.237.156]>
References: <3.0.3.32.19981203222234.0093ccc0@pop.xs4all.nl>
 <28da01be1ef8$ea915bb0$2d232cd1@linwin.token.hapenney.com>
Message-ID: <3.0.3.32.19981206213514.0093d980@pop.xs4all.nl>

At 23:38 98/12/04 +0100, Just van Rossum wrote:
[...]
>A proposal for a flat object hierarchy (a.k.a. "The Earth is Flat")
[...]

That's very much along the lines that I had in mind.

I have some observations:

1) Attribute retrieval can be defined much cleaner. You currently use
different rules to search the bases then you use to search the object. If a
base-object is an object, why not use the standard search system.

Proposal:

Getting an attribute is defined recursively. There are many ways to do
this, and I don't know if the one I've chosen is the best one. (No doubt
this needs more study.) I have split the functionality of getting an
attribute of the object itself, and getting an attribute for one of your
derived objects. This just seems a clean way to do things. I can imagine
that you might want to change the get_attribute behaviour of one object
without changing it for all derived objects too. (I've used my own name to
avoid clashes with existing ideas.)

ntf_get_attribute( object, name ):
	'''This is the standard get-an-attribute function.'''
	getfunc = ntf_get_attribute_raw( object, '__get_attribute__' )
	if getfunc != None:
		return getfunc( name )
	else: 
		return ntf_get_attribute_raw( object, name )

ntf_get_attribute_raw( object, name ):
	if object.__namespace__.has_key(name):
		return bound_attribute( object, object.__namespace__[name] )
	for i in range( len( object.__bases__ ) ):
		base = object.__bases__[i]
		base_get = ntf_get_attribute( base, '__get_derived_attribute__' )
		if base_get!=None:
			rebind, attr = base_get( name, object.__bases__[i+1:] )
			if attr != None:
				if rebind
					attr = rebind_attribute( object, attr )
				return attr
		else:
			attr = ntf_get_attribute( base, name )
			if attr != None:
				return bound_attribute( object, attr )
	return None

The first function provides the __get_attribute__ hook, which can be
provided either by the object itself, or by one of its bases. 

The second function does the actual searching. (It should probably be
available to any __get_attribute__ function to let it do its work.) It
searches first in the local namespace. If it finds an entry for the desired
name, it returns a bound attribute. If there is no local entry, the bases
are searched in turn. The exact details are not that important, but note
that each of the bases is only queried using the standard attribute access
mechanism. This ensures that the bases behave consistently, and that
attribute access doesn't change when you do it as a base class.

As an extra refinement, the 'get_derived_attribute' method (if present)
gets the rest of the 'bases' list. This allows a base in the list to
control how the other bases are used to create the object behaviour. We can
now create an object which I shall call 'Class' such that:

	class X( Y, Z ): [...]
is equivalent to
	object X( Class, Y, Z ): [...]

and the Class base can take care of the way in which the attributes of
object X  depend on the attributes of objects Y and Z.

I've tricked around a bit with the return value of get_derived_attribute.
It returns an extra boolean that specifies whether the attribute should be
re-bound to the derived object. This supports the run-time flexible version
of static class methods.

As specified this access rule is rather inefficient. It will probably
require some form of optimisation, which I will discuss later.

2) Why the named/unnamed distinction?
In your proposal you distinguish named and unnamed objects; one returns
bound methods and one unbound ones. Why not return bound methods all the
time? If you really want to, you can always re-bind it, or delete the
binding. You should certainly not use the named/unnamed distinction for
this. If you really need it, you should have a bound/unbound flag
somewhere. To get the full flexibility you would want to specify the
binding of each attribute separately, not for all the attributes in an
object in one go.

If you always return bound attributes, we might be able to get rid of the
entire idea of global things. After all, global items in a program are
basically bound to the program-invocation-object. (In this view, you don't
start a program, but you instantiate an object derived from the program
object. Thus, there can be several program invocation simultaneously.)

3) It is probably a good idea to put any named base objects in the local
namespace by default. The X.__init__ function wants to be able to call
self.Y.__init__, and thus self.Y must resolve to the right base object. We
can no doubt think of a couple of alternative ways of solving this, but
this seems the easiest.

4) I have not made a distinction between methods and other data elements.
All attributes that are returned are bound to an object from which they
came. Is this how Python works at the moment?



I think the ideas above support all the wishes that I have seen on this
mailing list (at least, those that I could understand). Delegation should
be straightforward. The biggest problem is that this system is more than
enough rope for anyone to hand herself.


Compatibility:
I'm mainly thinking about Python 2, but by choosing different special names
I think it can be made fully compatible with the existing Python 1.5.


Efficiency:
A straightforward implementation would probably be inefficient, some form
of optimisation is needed.

For the overwhelming majority of all programs, base objects don't change
their own behaviour during the lifetime of an object. 

A simple caching scheme is the following: Suppose we have object X, with
bases A, B, and C. If an attribute of X is retrieved from base B, the
resulting attribute is cached in X. The base object B is informed of this
fact, and keeps a link to X. Whenever that (or any) attribute of B is
changed, object X is informed and the cached reference removed. As the same
scheme is applied to all levels of the hierarchy, this will work without
problems.

No doubt we can all think of many other ways to improve the efficiency.
Global program analysis and optimisation is one technique that is extremely
powerful, but I'll leave that to another mail.

Niels


============================================================
Niels Ferguson, vorpal@xs4all.nl, voice/fax: +31 20 463 0977
PGP: 50E0 CBE2 3F19 C17D  FBE3 FFA5 38B7 BBB5


From sjmachin@lexicon.net  Sun Dec  6 22:25:19 1998
From: sjmachin@lexicon.net (John Machin)
Date: Mon, 7 Dec 1998 08:25:19 +1000
Subject: [Types-sig] Why have two hierarchies? (*EUREKA!*)
In-Reply-To: <000001be20f7$2bf46340$6b9e2299@tim>
References: <l03102809b28f72d70122@[193.78.237.160]>
Message-ID: <19981206213701100.AAA181.283@max4101.izone.net.au>

On 6 Dec 98, at 4:02, Tim Peters wrote:

> [Just van Rossum]
> > What is a DAG?
> 
> [Evan Simpson]
> > Directed Acyclic Graph,
> 
> [Just]
> > Yeah right, like that would help a poor graphic designer <wink>.
> 

[Tim's long winded but amusing diatribe snipped]

> 
> Dag's all, folks!
> 
> fatigue-dulls-the-pain-but-awakes-enticing-thoughts-of-death-ly y'rs  -
> dag

Try looking it up in a good dictionary: A dag is a small piece of 
dried excrement clinging to the wool on a sheep's posterior.

John Machin
1/27 Auburn Grove
Hawthorn East, VIC 3123, Australia
Phone: +61-3-98130561
mailto: sjmachin@lexicon.net


From just@letterror.com  Mon Dec  7 01:09:30 1998
From: just@letterror.com (Just van Rossum)
Date: Mon, 7 Dec 1998 02:09:30 +0100
Subject: [Types-sig] New article + demo
Message-ID: <l03102804b290d9a988dc@[193.78.237.166]>

Folks,

I've put a new version of my non-meta-article at:
     http://starship.skyport.net/~just/meta/
which links to a completely new demo.

It's all still very rough, although it's better than it was before. It
contains a possible solution for class methods. Some ideas of Evan have
been incorporated.

Many details need more thinking still...

Just




From just@letterror.com  Mon Dec  7 17:14:38 1998
From: just@letterror.com (Just van Rossum)
Date: Mon, 7 Dec 1998 18:14:38 +0100
Subject: [Types-sig] New article + demo
In-Reply-To: <l03102804b290d9a988dc@[193.78.237.166]>
Message-ID: <l03102807b291bb1348f0@[193.78.237.166]>

I've added two graphs (technically: Dumb Art Graphs) to the article. They
prove either:

    - that I'm wrong, since they show that my scheme is just
      as complex as one which does have "real" metaclasses.

    - that I'm right, in sofar that my scheme is equivalent to
      one which does have "real" metaclasses.

Or both. ;-)

Still at:
     http://starship.skyport.net/~just/meta/

Just




From Donald Beaudry <donb@tensilica.com>  Mon Dec  7 18:12:14 1998
From: Donald Beaudry <donb@tensilica.com> (Donald Beaudry)
Date: Mon, 07 Dec 1998 13:12:14 -0500
Subject: [Types-sig] Why have two hierarchies? (*EUREKA!*)
References: <3.0.3.32.19981203222234.0093ccc0@pop.xs4all.nl> <28da01be1ef8$ea915bb0$2d232cd1@linwin.token.hapenney.com> <l03102800b28ee586b8eb@[193.78.237.169]> <l03102805b28f30f07b30@[193.78.237.160]>
Message-ID: <199812071812.NAA05694@hq.tensilica.com>

Just van Rossum <just@letterror.com> wrote,
> What's the problem? ;-) (Seriously: I *may* just not get it: please
> elaborate...)

Perhaps there isnt one.  What you are suggesting reminds me quite a
bit of the type system used in Self.  Or maybe I should say that it
reminds me quite a bit of what I remember of the type system that is
used in Self.  I really need to do some reading... you could be on to
something.  (Since this is a transcontinental rant, I dont have net
access)

But, what I unsuccesfully tried to point out in my last message was
that all you are doing is renaming things.  You've proposed that what
we now call __class__ should be called __bases__ and what we now call
__dict__ should be called __namespace__.  You then proposed that we
stop referring to classes and metaclasses as such and simply refer to
them as instances.  I dont think that this renaming buys us anything.

When it comes right down to it there will be instances that must act
like the classes that we have now.  People will probably even start
calling them 'classes' to describe to others the role they are
playing.  I do like the idea that classes become more like instances
but I see no need for eliminating the term 'class'.  Likewise, as
classes become more like instances, they too will be associated with a
class object.  But, this class object will be quite a bit different
from the class objects we know today.  Rather than existing for the
purpose of creating new "instances", they will exist for the purpose
of creating new classes.

I can understand your desire to simplify this whole mess.  Some of
people reading this list are familar with my unsuccessful attempts to
explain what I consider to be a very elegant and simple concept.  The
concept being that all objects should be alike in some very
fundemental ways.  Unfortunately, the implementation of this
"everything is a turtle" approach has some interesting implications.
The are also some interesting implications associated with being a
raving lunatic who tries to explain simple concepts. (Yes Gordon, I've
given up on trying to hide it, you were right all along ;)

To be honest, I dont even understand the subject line in this thread.
What are these two hierarchies anyways?  Are you referring to the type
vs class hierarchies or the meta vs non-meta hierarchies?  Personally,
I dont belive in the need for either of these splits.

The type vs class issue is simply an artifact of the current
implementation.  Quite a while back, the MESS tried to show the world
that this was indeed the case.  In the MESS, both type objects and
class objects exist, but belong to the same heirarchy.  That is, class
objects are derived from type objects, therefore, a class object
*is-a* type object.  The useful distinction between the two objects is
that type objects are used to describe objects that have been
implemented in C while class objects are used to describe objects that
have been implemented in Python.

As for the meta vs non-meta hierarchies, they too are derived from a
common base so are in fact part of the same hierarchy.

I think too, that it's at least slightly interesting to point out that
from the start the Python implementation has recognized the need for
the existance of a "meta-entity" by way of the Typeobject.  For some
strange reason, the fact that the Typeobject is an instance of itself
has not been known for causing any heads to explode.  It's a very
simple and useful idea.  The mergeing of all of this into a single
style for the creation of objects is simply an extension of the
capabilities of the current Typeobject.

In my opinion, the interesting questions have to do with the interface
to the interpreter (the C API) and the interface to the user (the
Python syntax).  In the past, the primary complaints about my
proposals have centered around the notion that what I am asking for is
too powerful or too general, and therefore to difficult to explain and
understand.  While those issues dont bother me, I will agree that the
complaints are valid.  The excess generality should trimmed.  The
power should be restricted.  So, how do we do this?  What should be
do-able from the C API?  What should be do-able from Python itself?

Perhaps much of this has already been covered, if so, I'll apologize
now for not having read much of what has been posted to this list.
I'm trying to catch up.

	--Don



From just@letterror.com  Mon Dec  7 21:21:11 1998
From: just@letterror.com (Just van Rossum)
Date: Mon, 7 Dec 1998 22:21:11 +0100
Subject: [Types-sig] Why have three hierarchies? (*BOOM*)
In-Reply-To: <199812071812.NAA05694@hq.tensilica.com>
References: <3.0.3.32.19981203222234.0093ccc0@pop.xs4all.nl>
 <28da01be1ef8$ea915bb0$2d232cd1@linwin.token.hapenney.com>
 <l03102800b28ee586b8eb@[193.78.237.169]>
 <l03102805b28f30f07b30@[193.78.237.160]>
Message-ID: <l03102803b291e5a66f1b@[193.78.237.180]>

Donald ["The Hook"] Beaudry wrote:
>But, what I unsuccesfully tried to point out in my last message was
>that all you are doing is renaming things.  You've proposed that what
>we now call __class__ should be called __bases__ and what we now call
>__dict__ should be called __namespace__.  You then proposed that we
>stop referring to classes and metaclasses as such and simply refer to
>them as instances.  I dont think that this renaming buys us anything.

(I also noted that the choice for the name __namespace__ quite is
arbitrary: I just can't use __dict__ since anything stored in it will be
found by the current Python machinery before my own getattr is ever
invoked. And I just realized that __bases__ should be called __classes__.)

Naming issues aside, when we merge __class__ and __bases__ into one thing,
what could possibly be the meta class but the content of the mergee? Maybe
it's a cheap shot, but in my view it killed pretty effectively ;-). Ok,
that was excellent evidence proving my skull is very thick (not that that
prevents it from exploding, though).

Seriously, maybe you're right. If I read you correctly, the meta object of
my objects is really their builtin definition/implementation. But: if we
have only *one* of these meta objects, and we make it very versatile, there
should be no reason to redefine (ie. to subclass) it from Python. Maybe it
doesn't even need to be accessible from Python. Maybe it's not even a real
object. Maybe that's good: that way we can maybe keep implementation
details out of the language definition. Maybe that's just naive.

>I do like the idea that classes become more like instances
>but I see no need for eliminating the term 'class'.

I don't make classes more like instances, I make instances more like
classes (and that might even be the crux of my rant). And of course the
term class is not ready for the recycling bin. I have my doubt about the
word "meta class" though ;-).

[ ... ]
>To be honest, I dont even understand the subject line in this thread.
>What are these two hierarchies anyways?  Are you referring to the type
>vs class hierarchies or the meta vs non-meta hierarchies?  Personally,
>I dont belive in the need for either of these splits.

Well, you've created it yourself ;-). Maybe it's not really there, but
that's where my proposal kicks in: it is *really* not there... Seriously,
making classes have separate __bases__ and __class__ attributes at least
gives the impression of two hierarchies. And that's enough to make some
minds boggle. But I see what you mean.

Thanks much for the elaborate answer, it will probably take some more time
on my side to let it fully sink in. But even in this short time I've
learned a lot! Feels like I just had a free lunch.

Just




From just@letterror.com  Mon Dec  7 22:51:13 1998
From: just@letterror.com (Just van Rossum)
Date: Mon, 7 Dec 1998 23:51:13 +0100
Subject: [Types-sig] Why have two hierarchies? (*EUREKA!*)
In-Reply-To: <3.0.3.32.19981206213514.0093d980@pop.xs4all.nl>
References: <l03102802b28d86ad4077@[193.78.237.156]>
 <3.0.3.32.19981203222234.0093ccc0@pop.xs4all.nl>
 <28da01be1ef8$ea915bb0$2d232cd1@linwin.token.hapenney.com>
Message-ID: <l03102805b292060e1135@[193.78.237.180]>

Niels Ferguson wrote:
>I have some observations:
>
>1) Attribute retrieval can be defined much cleaner. You currently use
>different rules to search the bases then you use to search the object. If a
>base-object is an object, why not use the standard search system.

Ok.

>Proposal:
>
>Getting an attribute is defined recursively. There are many ways to do
>this, and I don't know if the one I've chosen is the best one. (No doubt
>this needs more study.) I have split the functionality of getting an
>attribute of the object itself, and getting an attribute for one of your
>derived objects. This just seems a clean way to do things.

Yes, that's roughly what I did, too.

>I can imagine
>that you might want to change the get_attribute behaviour of one object
>without changing it for all derived objects too. (I've used my own name to
>avoid clashes with existing ideas.)

(I've stolen your "raw" idea and sort of incorporated it in my stuff as
__raw_getattr__. I don't thing I've got it right yet, tho.)

[ many good ideas snipped ]

>2) Why the named/unnamed distinction?
>In your proposal you distinguish named and unnamed objects; one returns
>bound methods and one unbound ones. Why not return bound methods all the
>time? If you really want to, you can always re-bind it, or delete the
>binding. You should certainly not use the named/unnamed distinction for
>this. If you really need it, you should have a bound/unbound flag
>somewhere. To get the full flexibility you would want to specify the
>binding of each attribute separately, not for all the attributes in an
>object in one go.

You've got a great point here. My brain is too slow today to add anything
useful to it right now.

>4) I have not made a distinction between methods and other data elements.
>All attributes that are returned are bound to an object from which they
>came. Is this how Python works at the moment?

How do you bind an integer?

This is not how Python works right now. A class' __dict__ contains raw
objects. When accessed through a class or an instance the value's type is
checked, and wrapped in a method object if we're dealing with a function.
This happens on the fly: it prevents circular references
(class->function->class). If the requested object is an instance, the
method is additionally bound to the instance.

>Compatibility:
>I'm mainly thinking about Python 2, but by choosing different special names
>I think it can be made fully compatible with the existing Python 1.5.

An additional __raw_getattr__ special method to intercept all attribute
access seems a good idea anyway, no matter what direction Python 2.0 will
go.

(Vague ideas of how to do really neat automatic delegation using my scheme
are floating around in my head. Darn! They won't sit still.)

Just




From gmcm@hypernet.com  Mon Dec  7 23:17:34 1998
From: gmcm@hypernet.com (Gordon McMillan)
Date: Mon, 7 Dec 1998 18:17:34 -0500
Subject: [Types-sig] New article + demo
In-Reply-To: <l03102807b291bb1348f0@[193.78.237.166]>
References: <l03102804b290d9a988dc@[193.78.237.166]>
Message-ID: <1299067255-999812@hypernet.com>

Just van Rossum writes:

> I've added two graphs (technically: Dumb Art Graphs) to the article.
> They prove either:
> 
>     - that I'm wrong, since they show that my scheme is just
>       as complex as one which does have "real" metaclasses.
> 
>     - that I'm right, in sofar that my scheme is equivalent to
>       one which does have "real" metaclasses.
> 
> Or both. ;-)

Well your graphs don't really measure complexity. Nor is complexity 
really the major issue. Nor have we found out how complex things 
really get.

The important thing is unearthing and discussing (the more violently, 
the better <wink>) what we all expect out of a class model.

For example, you get rid of the difference between an instance and a 
subclass, then reimpose one simply for the sake of having one. 
(Whatsamatta, Just, you chicken? Huh??).

Disposing of that distinction (depending on how it's done) could 
have quite a few consequences. Conceptually, current Python 
fits with class models that say an intance is fundamentally 
different than a class. (Python's dynamic nature does make it 
somewhat more difficult to classify than most languages). Classes 
hold behavior, instances hold state, and an instance's class can be 
regarded as one, unified thing, created by munging together all of 
the bases. (The fact that __bases__ is a tuple of a bunch of 
different objects is an implementation detail; behavior is pretty 
much a done deal once __bases__ has been specified).

You are now holding state all through the hierarchy. Which might 
yield some unexpected results:

class A:
  def static_modifying_method(...)
    #this somehow modifies state

class X(A,B):
  #whatever

class Y(C, A):
   #whatever

y = Y()
r1=y.whatsit()   #results depend on state held in A

x = X()
x.static_modifying_method(...)  #state changed

r2=y.whatsit()  #hey, wait a minute - results are different!!!

To tell the truth (generally a bad idea on Usenet <wink>) I'm not 
quite sure what I'm whining about. Is it:

1) In current Python, all the mutable stuff (in the absence of 
sufficient perversity) is confined to the instance, which makes me 
feel safe and comfortable, (stop laughing, Tim!).

2) The fact that it appears this proposal will mean that the 
inheritance DAG will actually have something to do with the runtime 
structure. Coming from the C++ world, I'm used to inheritance being 
simply a way to structure code, not objects.

3) Confusion about "static" methods vs. "class" methods; what the 
latter are useful for, and if the 2 flavors can co-exist.

This deserves some discussion, actually. "Static" methods are 
generally justified as a way to hold code in a sensible place - they 
do something related to secret knowledge the class has, but don't 
need an instance. This usage registers about a .01 on my personal 
Richter scale. I use static methods in C++, but to me they're a "so 
what".

I use "static" in C++ to create singletons - make all the state 
static, and all the instances you create are really the same 
instance. Hee, hee. Very handy. Due to the syntax of C++, if I 
subclass one of these guys, I get a completely independent (set of) 
singletons.

Is this what "class" methods (with a __class_self__) are for? How do 
they behave when subclassed?

I'm also concerned with the installation of __getattr__ hooks. You're 
currently looking for *some* __getattr__ hook in the bases, and using 
that. But is that a depth first search? Or just a search of 
__bases__? If I have:

class A(Trace):
  #blah

class B(A)
  def mymethod(self, arg): #blah

b = B()

Will "b.mymethod(1)" trace?

And more generally (in anybody's scheme), how do you handle multiple 
__getattr__ hooks? Just take the first? The last? Let one wrap the 
attribute and hand it to the next to wrap again?

discussing-behaviors-at-the-edge-of-a-black-hole-ly y'rs

- Gordon


From just@letterror.com  Tue Dec  8 00:47:00 1998
From: just@letterror.com (Just van Rossum)
Date: Tue, 8 Dec 1998 01:47:00 +0100
Subject: [Types-sig] Instances vs. Classes [vs. Types]
In-Reply-To: <1299067255-999812@hypernet.com>
References: <l03102807b291bb1348f0@[193.78.237.166]>
 <l03102804b290d9a988dc@[193.78.237.166]>
Message-ID: <l0310280bb29218465bf1@[193.78.237.180]>

Gordon McMillan wrote:
>For example, you get rid of the difference between an instance and a
>subclass, then reimpose one simply for the sake of having one.
>(Whatsamatta, Just, you chicken? Huh??).

I'm afraid of losing the Python I love. I'd hate to call the neighbors and
ask whether they've seen it. On the other hand, if I were a chicken, I'd be
Python poop by now.

>Disposing of that distinction (depending on how it's done) could
>have quite a few consequences. Conceptually, current Python
>fits with class models that say an intance is fundamentally
>different than a class. (Python's dynamic nature does make it
>somewhat more difficult to classify than most languages). Classes
>hold behavior, instances hold state, and an instance's class can be
>regarded as one, unified thing, created by munging together all of
>the bases. (The fact that __bases__ is a tuple of a bunch of
>different objects is an implementation detail; behavior is pretty
>much a done deal once __bases__ has been specified).

Ok, I'm here.

[ ... ]
>To tell the truth (generally a bad idea on Usenet <wink>) I'm not
>quite sure what I'm whining about. Is it:
>
>1) In current Python, all the mutable stuff (in the absence of
>sufficient perversity) is confined to the instance, which makes me
>feel safe and comfortable, (stop laughing, Tim!).
>
>2) The fact that it appears this proposal will mean that the
>inheritance DAG will actually have something to do with the runtime
>structure.

Well, it's possible to muck with it at runtime now, too:

	class A: pass
	class B: pass
	print B.__bases__
	B.__bases__ = (A,)
	print B.__bases__

>Coming from the C++ world,

Talk to Andrew Dalke, he goes to these meetings that seem to help him a lot.

>I'm used to inheritance being
>simply a way to structure code, not objects.

It's the same in Python, but resolving is done at runtime. Python's
slowness has to come from *something*, doesn't it? As Guido mentioned that
Python 2.0 will probably initially be even slower than now, we might as
well help him a little in materializing that goal by demanding rediculously
expensive attribute lookup schemes.

>3) Confusion about "static" methods vs. "class" methods; what the
>latter are useful for, and if the 2 flavors can co-exist.
[ ... ]
>Is this what "class" methods (with a __class_self__) are for?

No, they're class methods: they work on class objects.

>How do they behave when subclassed?

That's the issue. I don't know yet. My proposal sucks eggs big time in this
area.

>I'm also concerned with the installation of __getattr__ hooks. You're
>currently looking for *some* __getattr__ hook in the bases, and using
>that. But is that a depth first search? Or just a search of
>__bases__?

Depth first: Python's standard behavior. It should take the first it finds.
But since the version of the article you've read I've changed my mind. I
called it __raw_getattr__ instead and that one will get called for all
attribute access if available. The old and trusty __getattr__ could do the
same it does now: as a fallback when the builtin mechanism has failed. That
means if you want a __raw_attr__ *and* a __getattr__, your __raw_getattr__
is the boss: it has to take care of finding and executing the custom
__getattr__. If it wants. It's the boss.

>If I have:
>
>class A(Trace):
>  #blah
>
>class B(A)
>  def mymethod(self, arg): #blah
>
>b = B()
>
>Will "b.mymethod(1)" trace?

Of course! Unless B implements it's own __raw_getattr__ that _doesn't_ trace.

>And more generally (in anybody's scheme), how do you handle multiple
>__getattr__ hooks? Just take the first? The last?

If we have an inheritance graph (technically, a Deep Acid Graph) like this:

	D(C(B(A)))   ie. B is a subclass from A, etc.

C might define a raw_getattr, and D not. A defines one, too, but B not.
If we ask D for an attribute, it will call C's raw_getattr, which might
want to ask B for an attribute, which will invoke A's raw_getattr. Etc.

This sounds more inefficient than it is: we have to recursively traverse
the bases of each anyway, probably by calling some function. Whether that
function is the same one or a different one each time doesn't matter that
much. Of course a C implementation will always beat a Python version hands
down, and the more tricks like this you do, the more you will pay. These
raw_gettr methods could obviously be cached, although that sacrifices some
dynamic-ness: what if we want to muck with __bases__? (And I do...) So
you're right. It stinks.

And what will happen if I do this:

class A:
    def __raw_getattr__(self, attr_): # blah

class B:
    # no __raw_getattr__
    def foo(self): # blah

a = A()
b = B()

a.__bases__.append(b)  # cheap delegation!?

???

That's an idea I have: since any object can have more bases, instances, can
have more bases, too. In the above example I may want to fetch methods
through 'a', which belong to 'b', therefore could be bound to 'b'! Using my
__name__ artifact (shut it Gordon!) we could maybe stop re-binding in the
getattr chain at the last nameless object. a.foo() would be bound to b.
Catch my drift?

>Let one wrap the
>attribute and hand it to the next to wrap again?

Maybe not wrap again but rather _bind_ again.

everything-is-a-getattr-function-ly y'rs --  just




From Edward Welbourne <eddyw@lsl.co.uk>  Tue Dec  8 12:21:19 1998
From: Edward Welbourne <eddyw@lsl.co.uk> (Edward Welbourne)
Date: Tue, 8 Dec 1998 12:21:19 GMT
Subject: [Types-sig] method binding
Message-ID: <199812081221.MAA02746@lsls4p.lsl.co.uk>

All values are objects, including the objects (classes) which describe
the behaviours of other objects (their instances).  One object can `use'
another as `its class' and it can use others in many other ways.

We have three distinct kinds of deferral for attribute lookup at present:
the object (having failed to find the attribute in its own namespace)

 i) asks its bases if any of them has the attribute: if so, it treats
    this just as if it'd found it in its own namespace.  At present,
    only classes do this, but all objects should be entitled to use
    .__bases__ for implicit deferral in the same way.

 ii) asks its class to give it such an attribute: the class looks up the
    attribute on itself and offers that for the job, possibly binding
    this answer as a method bound to the object which asked.

 iii) looks for a __getattr__ attribute of its own and calls that.  This
    can do smarter things than just `if it makes sense to bind the
    result I would be giving, I will'.

So, Just, I reject the unification of bases and class.

The bases of an object are just some places where the object looks for
uncompilicated values, with no `maybe binding' rule.  Things it finds
there are treated just the same way as things it finds in its own
namespace.  This is what we see at present if we ask for an attribute of
a class and get our answer from one of its bases.  So __bases__ provides
for delegation, only nobody'd commented on it.

If the object doesn't find the attribute in its own namespace or in its
bases, it asks its class/type/kind/whatever to bail it out.  At this
point we're doing something entirely different from bases: we're asking
someone else to generate a value, by whatever means, suitable for use as
the object's relevant attribute.  Both i) and iii) are like this.  This
is where `maybe binding' happens.

So I reckon our objects have got:

 a (personal) namespace, .__dict__
 a DG of bases, .__bases__
	(I'm not assuming my Directed Graph is acyclic)
 a sequence of assistors, .__classes__

and attribute lookup works like this:

def getattr(object, key):
	try: return object.__dict__[key]
	except KeyError: pass
	# if key in ( '__bases__', '__classes__', '__getattr__',
	# '__getinstattr__' ): raise AttributeError, 'perhaps'

	for base in object.__bases__:
		try: return getattr(base, key)
		except AttributeError: pass

	for helper in object.__classes__:
		try: meth = getattr(helper, '__getinstattr__')
		except AttributeError: pass
		else:
			try: return meth(object, key)
			except AttributeError: pass

	if key != '__getattr__':
		# beware the unhandled recursion !
		try: meth = getattr(object, '__getattr__')
		except AttributeError: pass
		else:
			try: return meth(object, key)
			except AttributeError: pass

	raise AttributeError, key

subject, of course, to the name lookups that the above does in the style
ob.name all being ones that some suitable magic can do without having to
call getattr.

Note that `maybe binding' activities are entirely the business of the
__getinstattr__ defined by something our object is using as a class and
of the __getattr__ of the object itself.  If you want to get a bound
method out of the __dict__ or __bases__ lookup, you have to have stored
the attribute as a bound method in the first place (and, then, it's
bound to whichever base you bound it to, if you bound it to a base).

So: there's no division into two kinds of entity, there's just a
division into two ways that one object can use others as a means of
getting hold of attributes.  In python, `a means of getting hold of
attributes' is support for the getattr interface: and support for
interfaces is entirely defined in terms of getting hold of attributes.
Which is why we're talking about attribute munging rather than any of
the other interfaces: it is the base of all interfaces.

There's also no benefit in doing fancy and different things based on
whether or not the object whose attribute we're returning has got a name
or class attribute.  These are irrelevant.  All the method binding is
contained in:

class Class:
	"""An object for use as a base by objects being used as classes.

	Defines the basic __getinstattr__ that a class provides for its
	instances. """

	def __getinstattr__(self, obj, key):
		meth = getattr(self, key)
		try: return meth.bind(obj)
		except BindingError: return meth

though, of course, whenever getattr() falls back on the object's
.__getattr__, recursing on itself, the result it got back may have been
bound to the object; and this method may, in turn, chose to bind the
result *it* will be returning likewise.

Must get on with some work now,

	Eddy.


From Edward Welbourne <eddyw@lsl.co.uk>  Tue Dec  8 12:28:35 1998
From: Edward Welbourne <eddyw@lsl.co.uk> (Edward Welbourne)
Date: Tue, 8 Dec 1998 12:28:35 GMT
Subject: [Types-sig] Re: method binding
In-Reply-To: <199812081221.MAA02746@lsls4p.lsl.co.uk>
References: <199812081221.MAA02746@lsls4p.lsl.co.uk>
Message-ID: <199812081228.MAA02771@lsls4p.lsl.co.uk>

Doh.  Dumb error:

> At this point we're doing something entirely different from bases: ...
> Both i) and iii) are like this.  This is where `maybe binding'
> happens.

for i) read ii), of course.

And some of the ob.name lookups in getattr() might be candidates for
getattr() to do ... but not all !

	Eddy.


From Edward Welbourne <eddyw@lsl.co.uk>  Tue Dec  8 13:57:05 1998
From: Edward Welbourne <eddyw@lsl.co.uk> (Edward Welbourne)
Date: Tue, 8 Dec 1998 13:57:05 GMT
Subject: [Types-sig] method binding
In-Reply-To: <199812081221.MAA02746@lsls4p.lsl.co.uk>
References: <199812081221.MAA02746@lsls4p.lsl.co.uk>
Message-ID: <199812081357.NAA07248@lsls4p.lsl.co.uk>

Hmm.  I think the important bit of all that was:

There is a difference between
   that object uses this one as a `base'
   that object uses this one as a `class'

In the first case, that one asks this one for an attribute on exactly
the same terms as it was, itself, asked for an attribute.

In the second, the instance asks its class to `work out' the attribute
for it and the class may well do some fancy things (like binding) in the
process of filling the request.  In particular, the class may do `first
type' attribute lookup `on itself' to get the thing it uses to build the
attribute it contributes to the instance.  For these purposes, there
doesn't really seem to be any fundamental difference between an object's
class and the object.__getattr__() method.

So we really need *two* kinds of `get attribute' functionality:
    plain raw `get my attribute'
    build attribute `for my instance'

But now to wait for all those huge brains to assimilate my little
tuppence worth ;^)

	Eddy.


From evan@tokenexchange.com  Tue Dec  8 18:03:41 1998
From: evan@tokenexchange.com (Evan Simpson)
Date: Tue, 8 Dec 1998 12:03:41 -0600
Subject: [Types-sig] method binding
Message-ID: <006801be22d5$17bc2fe0$2d232cd1@linwin.token.hapenney.com>

Some twist!  Let me see if I've got all the parts.

1. No object is intrinsically a class.  "Classness" is a way of *using* an
object.

2. Objects can delegate attribute fetching to __bases__ objects in almost
the same way current classes do, except that the fetched attribute is not
processed or hooked in any way (eg. you get raw functions instead of
methods).

3. Objects can also delegate attribute fetching to __classes__ objects, or
more specifically, to their __getinstattr__ functions.

4. If all else fails, the object can use its own __getattr__ to produce an
attribute.

Mmf.  This really turns things on their heads.  JvR's scheme essentially
leaves classes untouched, alters instances slightly, and provides some new
opportunities for hooks and anonymous objects.  Existing code works
unchanged.  This, on the other hand, would break everything.  Not a deadly
objection, if we're thinking Python 2.0, but still significant.

How do you spell something which acts like classes do now?  How do you
obtain an 'instance' from this 'class'?  How do you choose to treat an
object as a class when handling it directly (not through delegation from an
'instance')?

I'm confused, but interested.

Evan



From Edward Welbourne <eddyw@lsl.co.uk>  Tue Dec  8 19:06:25 1998
From: Edward Welbourne <eddyw@lsl.co.uk> (Edward Welbourne)
Date: Tue, 8 Dec 1998 19:06:25 GMT
Subject: [Types-sig] method binding
In-Reply-To: <006801be22d5$17bc2fe0$2d232cd1@linwin.token.hapenney.com>
References: <006801be22d5$17bc2fe0$2d232cd1@linwin.token.hapenney.com>
Message-ID: <199812081906.TAA09561@lsls4p.lsl.co.uk>

> 2. Objects can delegate attribute fetching to __bases__ objects in almost
> the same way current classes do, except that the fetched attribute is not
> processed or hooked in any way (eg. you get raw functions instead of
> methods).

No `except ...' clause.  Not `almost': exactly.

class A:
	def fred(self): pass
class B(A): pass

B.fred isn't processed or hooked in any way, it just used the B->A bases
link.  It is exactly the same entity as A.fred.  It is not
	lambda *args: apply(A.fred, (B,) + args)

By contrest, after

b = B()

b.fred is munged, but that's because the lookup went via the b->B class link.

> 3. Objects can also delegate attribute fetching to __classes__ objects, or
> more specifically, to their __getinstattr__ functions.

That specific is important.  It would probably make sense to replace the
__classes__ list I involved with the list of __getinstattr__ callables.
Of course, these could be bound methods (enabling them to access the
namespace of whoever they're bound to, typically what we used to think
of as a class).

> This, on the other hand, would break everything.

;^>

> How do you spell something which acts like classes do now?
Just the way we do at present, I'd reckon.

> How do you obtain an 'instance' from this 'class'?

sorry, I missed part of the definition of

class Class:
	# stuff from before ignored

	def __call__(self, *args):
		# need builtin newobject function of course
		result = newobject()
		# result.__fetchers__ = (self.__getinstattr__,)
		result.__classes__ = (self,)
		try: meth = getattr(self, '__init__')
		except AttributeError: pass
		else: apply(meth, (result,) + args)
		return result

Anything subclassed from that base will then behave just like the sorts
of classes we're used to.  I imagine.

> How do you choose to treat an object as a class when handling it
> directly (not through delegation from an 'instance')?

I haven't understood this question.
So the following may be off topic.

`to treat an object as a class' is simply to have it in one's __classes__
list: there is no other sense in which one treats anything as a class.

> I'm confused, but interested.
Oh good: I've done something useful today ;^>

	Eddy.


From gmcm@hypernet.com  Tue Dec  8 20:30:26 1998
From: gmcm@hypernet.com (Gordon McMillan)
Date: Tue, 8 Dec 1998 15:30:26 -0500
Subject: [Types-sig] Meta-stuff
Message-ID: <1298990880-988548@hypernet.com>

Gordon's Predicated Provisional Proposed MetaClass Pre-Proposal (
Puh-puh-Proposed MC puh-Proposal ), or, How To Punt on the Theory
Stuff:

Observation 1: People use metaclasses to do getattr hooks.

Observation 2: Getattr hooks come in 3, no, 4 flavors:

  a) None (Surprise!)

  b) Vanilla (magic invoked when attr not found by normal means)

  c) Some attrs hidden from normal means, so magic invoked even 
     though attr is in the normal search path.

  d) Complete rewrite of the search mechanism.

Observation 3: Flavors (a) and (b) are normal Python. Flavors (c) and 
(d) are what current metaclasses are used for.

Observation 4: Usage of current metaclass hooks causes Spontaneous 
Brain Explosions.

Goal: Reduce SBEs by 80%.

Technique: Make the attribute search mechanism a Framework, with the 
possibiity to intervene at selected points.

Current mechanism:

  1. if name is "__dict__" or "__class__", return those

  2. look in instance.__dict__

  3. do a class_lookup:

     3a check class __dict__

     3b for base in bases: do a class_lookup

  4 if found:

    4a if a function, bind as method

    4b if a method, rebind

    4c return it

  5 if there's a getattr hook, call it, return the result

Now let's distinguish between "get" (the whole search starting from 
instance through class and bases to the magic __getattr__), from 
"Get" (which is the recursive class_lookup routine), from "GET" which 
is the raw look in a dictionary-or-moral-equivalent routine. (The 
lower the level, the upper the case.) And then there's 
"__GetNonExisting__", or the current __getattr__ hook.

So, very roughly we have:

get(inst, name) is:
  found, obj = GET(inst, name)
  if not found:
    found, obj = Get(inst.__class__, name)
    if not found:
      hook = get(inst, "__GetNonExisting__")
      if hook:
        found, obj = apply(hook, ...)
  if not found:
    raise AttributeError, ...
  if isFunction(obj):
    obj = bind(...)
  return obj
  
and Get(obj, name) is
  found, obj = GET(obj, name)
  if not found:
    for base in obj.__bases__:
      found, obj = Get(base, name)
      if found: break
  return found, obj

and GET(obj, name) is (usually):
  try: attr = obj.__dict__[name]
  except: return (0, None)
  else: return (1, attr)

Now lets turn that into a Framework. 

  1) The object has a chance to override GET (his attributes may not 
     be in a dict name __dict__; indeed, maybe not in a dict at all). 

  2) The class-like object which is the root of the branch being 
     searched has a chance to specify a new search pattern (ie, 
     override "Get"). 

  3) The root class-like object can intervene in any of the above, or 
     in the way the instance is searched (ie, override "get").

Penultimately, let's use a callback whenever an attribute matching 
"name" is found. The callback can say "dandy, we're done", or it can 
say "keep looking". In fact, the callback is responsible for whatever 
binding, wrapping or perversity that needs to be performed. So what 
gets returned to the caller of "get" is actually what the callback 
has picked, accumulated or munged. So, conceivably, by overriding the 
callback, we could chain together all the base class methods of the 
same name (this is a perversity that was asked for on c.l.p. by 
someone who shall remain unnamed - but I could give you a hint, 
if he doesn't pay me off first...).

This scheme also makes it possible to (relatively) straightforwardly 
pull tricks that currently require hiding the real attribute, so the 
magic hook will get invoked.

Also, "bind" is part of the framework, so it can be overridden 
without horsing around with the other stuff.

Finally, let's provide a default implementation of the 
framework that behaves like current Python. Call it the default 
Turtle. Let classes (instances, too?) specify their Turtles if they 
so desire.

Caveat: I have not explicitly dealt with how to handle multiple 
Turtles in the search. I think, with properly executed callbacks and 
temporary objects, it's possible to let them wrap each other's 
results. Whether the resultant object works as expected is another 
question.

Cop-out: I have some REAL work to do; I've hardly touched Just's 
stuff or JimF's stuff, so don't hold your breath waiting for a 
prototype implementation of the PPPMCPP.

and-th-th-th-that's-all-folks-ly y'rs

- Gordon


From gstein@lyra.org  Wed Dec  9 00:52:14 1998
From: gstein@lyra.org (Greg Stein)
Date: Tue, 08 Dec 1998 16:52:14 -0800
Subject: [Types-sig] Meta-stuff
References: <1298990880-988548@hypernet.com>
Message-ID: <366DC9BE.40B30AAB@lyra.org>

Gordon McMillan wrote:
> 
> Gordon's Predicated Provisional Proposed MetaClass Pre-Proposal (
> Puh-puh-Proposed MC puh-Proposal ), or, How To Punt on the Theory
> Stuff:
> ...

Ah. Finally something rational and understandable!

I do have to say that the traffic over the past week has been quite
weird. I watched as people went on about a lot of theory and just kept
asking myself, "why? what's the point?"

Did I miss it, or has somebody created a specific list of things to
change in the Python object model? In other words, what/why needs to
change? [Gordon makes a good argument point here: it is really just
getattr behavior]

Cheers,
-g

--
Greg Stein, http://www.lyra.org/


From Donald Beaudry <donb@tensilica.com>  Wed Dec  9 01:01:27 1998
From: Donald Beaudry <donb@tensilica.com> (Donald Beaudry)
Date: Tue, 08 Dec 1998 20:01:27 -0500
Subject: [Types-sig] Meta-stuff
References: <1298990880-988548@hypernet.com> <366DC9BE.40B30AAB@lyra.org>
Message-ID: <199812090101.UAA30424@hq.tensilica.com>

Greg Stein <gstein@lyra.org> wrote,
> Did I miss it, or has somebody created a specific list of things to
> change in the Python object model?

If you missed, you are not the only one.  But, then, I havnt been
following this list as closely as I would like to.

> In other words, what/why needs to change? [Gordon makes a good
> argument point here: it is really just getattr behavior]

There really are two different dimensions here that need to be
explored.  One could easily be viewed as getattr behavior.  I like to
think about it as the "Python view" on this discussion.  The other
issue is dealing with the unification of classes and types.  But, this
is really just an API issue.  A while back, there was some discussion
on what this might mean.

	--Don



From Edward Welbourne <eddy@chaos.org.uk>  Wed Dec  9 02:36:22 1998
From: Edward Welbourne <eddy@chaos.org.uk> (Edward Welbourne)
Date: Wed, 9 Dec 1998 02:36:22 +0000 (GMT)
Subject: [Types-sig] getattr glitch
Message-ID: <m0znZUE-00000OC@vortigen.demon.co.uk>

I noticed a glitch in my getattr, I now want it to start

def readattr(obj, key):
    try: return obj.__dict__[key]
    except KeyError: pass

def basereadattr(obj, key):
    try: return readattr(obj, key)
    except AttributeError: pass
    for base in obj.__bases__:
	try: return readattr(base, key)	# not getattr, I think.
	except AttributeError: pass

    raise AttributeError

def getattr(obj, key):
    try: return basereadattr(obj, key)
    except AttributeError: pass

    for whatever in obj.__classes__: ...


but this just makes basereadattr another simple and easy tool that we
can fit in with the whatever.__getinstattr__ methods and, for that
matter, the trailing __getattr__ protocol.

We'd then reduce to

def getattr(obj, key):

    for col in obj.__protos__:
	try: return col(obj, key)
	except AttributeError: pass

    raise AttributeError

and each col would `own' a piece of obj's namespace (__bases__ and
__dict__ owned by readattr and co-operating protocols, __getitem__ by
the fallback protocol in which __protos__ usually ends, etc.).
The familiar class/instance protocol (which owns __class__) becomes

def getinstattr(obj, key):
    val = getattr(obj.__class__, key) # fail with AttributeError if absent
    try: meth = val.bind(obj)
    except (AttributeError, BindingError): return val
    else: return meth

and appears in every instance's __proto__ just behind basereadattr.

Now __bases__ becomes redundant by putting
	lambda s, k, b=base, r=readattr: r(b, k)
into __protos__ for each base we would have put in __bases__.  One could
even use getattr for some bases, as I did in my original for all.

<snip>
It's too am and I wrote a whole lot more but I've snipped it off to make
sense of it later and I'll send the sane bit now <yawn>

      Eddy.


From evan@tokenexchange.com  Wed Dec  9 14:22:02 1998
From: evan@tokenexchange.com (Evan Simpson)
Date: Wed, 9 Dec 1998 08:22:02 -0600
Subject: [Types-sig] method binding
Message-ID: <00b101be237f$4aa8d570$2d232cd1@linwin.token.hapenney.com>

>> 2. Objects can delegate attribute fetching to __bases__ objects in almost
>> the same way current classes do, except that the fetched attribute is not
>> processed or hooked in any way (eg. you get raw functions instead of
>> methods).
>
>No `except ...' clause.  Not `almost': exactly.


Ah.  This is why I do these point-by-point summaries.  I seem to have
misread the following (from your original article):

>The bases of an object are just some places where the object looks for
>uncompilicated values, with no `maybe binding' rule.  Things it finds
>there are treated just the same way as things it finds in its own
>namespace.  This is what we see at present if we ask for an attribute of
>a class and get our answer from one of its bases.  So __bases__ provides
>for delegation, only nobody'd commented on it.

I took "no `maybe binding' rule" to mean that functions aren't wrapped as
methods or bound to an instance.

>By contrest, after
>
>b = B()
>
>b.fred is munged, but that's because the lookup went via the b->B class
link.
>
>> 3. Objects can also delegate attribute fetching to __classes__ objects,
or
>> more specifically, to their __getinstattr__ functions.
>
>That specific is important.  It would probably make sense to replace the
>__classes__ list I involved with the list of __getinstattr__ callables.

This confuses me even more.  It seems that you are saying that "fred" access
through "b" is automatically shunted to the "fred" attribute of "B", as
though "B" were a base of "b", not a class.  "B" certainly doesn't seem to
be or have a callable which could find and return "fred"s value.

>> How do you spell something which acts like classes do now?
>Just the way we do at present, I'd reckon.
>
[snip]
>> How do you choose to treat an object as a class when handling it
>> directly (not through delegation from an 'instance')?
>
>I haven't understood this question.
>So the following may be off topic.


I'm really not sure how your proposal differs from current Python anymore,
except possibly in allowing an instance to have multiple classes (how?).
You *have* suggested that attributes are fetched from an object in different
ways depending on whether they are found in a __classes__ search or a
__bases__ search (I think).  Thus we get "fred" from "B" differently if "B"
is a base of "b" than if it is a class of "b".  Given that, how does one go
about accessing an attribute of "B" DIRECTLY (not through "b") in these two
different manners?  Only be calling B.__get*__?



From Edward Welbourne <eddyw@lsl.co.uk>  Wed Dec  9 17:04:27 1998
From: Edward Welbourne <eddyw@lsl.co.uk> (Edward Welbourne)
Date: Wed, 9 Dec 1998 17:04:27 GMT
Subject: [Types-sig] method binding
In-Reply-To: <00b101be237f$4aa8d570$2d232cd1@linwin.token.hapenney.com>
References: <00b101be237f$4aa8d570$2d232cd1@linwin.token.hapenney.com>
Message-ID: <199812091704.RAA13565@lsls4p.lsl.co.uk>

>> No `except ...' clause.  Not `almost': exactly.
> Ah.  This is why I do these point-by-point summaries.  I seem to have
> misread the following (from your original article):
>> ...
> I took "no `maybe binding' rule" to mean that functions aren't wrapped as
> methods or bound to an instance.

and that was exactly what I meant.  The value an entry in an object's
__protos__ gives back to getattr is the value getattr'll use for the
attribute it was asking to see at the time.  Of course, the entry in the
object's __protos__ may be something which *did* bind the attribute to
the object, but that's none of getattr's business:

  getattr doesn't wrap or bind methods etc *ever* *at all*

but, of course, the __protos__ it invokes may be any manner of weird and
whacky stuff which might make vanilla icecream out of the object whose
attribute we wanted and return that as the attribute.  But getattr just
passes the result on as the attribute.

def readattr(obj, key):
	try: return obj.__dict__[key]
	except KeyError: raise AttributeError, key

def _getattr(obj, key):
	if key == '__getattr__': raise AttributeError, key
	fetch = getattr(obj, '__getattr__')
	return fetch(key)

Class.__protos__ = [
	readattr,	# so Class can look in its __dict__
	_getattr ]	# useless unles Class.__dict__.has_key('__getattr__')

getinstattr = Class.__getinstattr__.bind(Class)	# I seem to need this

First, Class has a __dict__ which contains __getinstattr__, which
readattr will find, so we get hold of this function expecting three
args: we bind it to Class to get getinstattr.

A.__protos__ = [
	readattr,	# A can read its __dict__, which has_key('fred')
	lambda o,k,g=getinstattr: g(o,k),	# A is of class Class
	_getattr ]

So A has 'fred' from its __dict__ the easy way.  If we look for
A.__getinstattr__, we'll be calling getinstattr(A, '__getinstattr__'),
which amounts to calling Class.__getinstattr__(Class, A, '__getinstattr__')
which finds Class.__getinstattr__ and binds it to A (successfully),
yielding the bound method A.__getinstattr__ which serves instances of A
the same way that getinstattr serves instances of Class.

B.__protos__ = [
	readattr,	# useless because B's __dict__ is empty
	lambda o,k,d=A,r=readattr: r(d,k),	# delegate to baseclass A
	lambda o,k,g=getinstattr: g(o,k),	# B is a Class
	_getattr ]

B gets a .__getinstattr__ the same way A did (note that its delegation to A
uses readattr, not getattr, so it doesn't see A.__getinstattr__): the
result is a bound method, with B as the class, waiting for obj and key args.
B also gets .fred from readattr(A, 'fred'): because this was readattr,
this is just A.fred without any fanciness.

b.__protos__ = [
	readattr,
	lambda o,k,g=B.__getinstattr__: g(o,k)	# b is a B
	_getattr ]

When we look up b.fred, we ask B.__getinstattr__(b, 'fred') which is
just Class.__getinstattr__(B, b, 'fred'), which calls getattr(B, 'fred')
and gets B.fred, which __getinstattr__ then duly binds to f.

If we ask for b.__getinstattr__, we call B.__getinstattr__(b, '__getinstattr__')
which is Class.__getinstattr__(B, b, '__getinstattr__'), which asks for
val = B.__getintattr__, which it finds, and then tries to val.bind(b),
which probably fails so we return B.__getinstattr__ as
b.__getinstattr__.  Of course, if we do some magic which lets it do some
further binding, this bind might succeed, leaving us with
	lambda k: Class.__getinstattr__(B, b, k)

Did that help the clarity any ?

> I'm really not sure how your proposal differs from current Python anymore,
it's much more flexible ?

I've only introduced the sorts of entries in __protos__ that would
implement the behaviours we're used to: but there is nothing to stop us
putting all manner of weird stuff into __protos__ to implement other
attribute fetching schemes not currently feasible in python.

It's also unfinished in the sense that I suspect it's unbootstrappable,
at least in its present form.

> You *have* suggested that attributes are fetched from an object in
> different ways depending on whether they are found in a __classes__
> search or a __bases__ search

well, not `on whether they are found in a ... search' but `on the
fetching tool that succeeded', which may be emulating the instance/class
protocol or the class/base protocol, in which case the tool doing the
job has done it differently, which is its right.

	getattr, on the other hand, does *no* munging,

no matter what proto gives it an answer, it takes that answer.

For B, the proto which gave it B.fred was readattr, which does nothing
special.  For b, the proto which gave it b.fred was B.__getinstattr__,
which does do something special.  That's all the difference.  If the
proto that gives me some attribute does hairy things, that's its
business.  It *isn't* getattr's business.

Passing thought:

class Class:
	def __getinstattr__(self, obj, key):
		if key == '__class__': return self

		val = getattr(self, key)
		try: return val.bind(obj)
		except (AttributeError, BindingError): return val

this would give the object a sensible __class__ attribute if its
__dict__ didn't have key '__class__'.  It would also ensure that an
object whose __dict__ doesn't have a __class__ is going to find the
class that first serves it as a class (in its __protos__ list) - rather
than the __class__ attribute of that first class ...

	Eddy.


From Edward Welbourne <eddy@chaos.org.uk>  Wed Dec  9 22:11:14 1998
From: Edward Welbourne <eddy@chaos.org.uk> (Edward Welbourne)
Date: Wed, 9 Dec 1998 22:11:14 +0000 (GMT)
Subject: [Types-sig] a toy
Message-ID: <m0znrpC-0000FYC@vortigen.demon.co.uk>

class Toy:

    def __init__(self, how):
	self.__protos__ = how

    def __getattr__(self, key):
	for col in self.__protos__:
	    try: return col(self, key)
	    except AttributeError: pass

	raise AttributeError

def toy(what, how):

    # provide our object with read-only access to what
    # under the name __datum__.
    def fetch(obj, key, dat=what):
	if key == '__datum__': return dat
	raise AttributeError, key

    how = [ fetch ] + how

    return Toy(how)

def namespace(what=None, how=[]):
    if what == None: what = {}

    def getit(obj, key, bok=what): return bok[key]
    def setit(obj, key, val, bok=what): bok[key] = val
    def delit(obj, key, bok=what): del bok[key]

    book = { '__dict__' : what,
	     # we could leave __dict__ out, of course ;^>
	     '__getattr__': getit,
	     '__setattr__': setit,
	     '__delattr__': delit }

    def bokit(obj, key, bok=book):
	try: return bok[key]
	except KeyError: raise AttributeError, key

    how = [ bokit ] + how

    return Toy(how)

def klass(what, bases, how=[]):
    row = []
    for base in bases:
	try: row.append(b.__getattr__)
	except AttributeError: pass

    return namespace(what, row + how)

def instance(C, *args):
    def getinstat(obj, key, c=C):
	val = c.__getattr__(key)
	try:
	    if not (val.im_self is None): return val
	except AttributeError: pass
	else: val.im_self = obj
	return val

    inst = namespace({}, [ getinstat ])
    try: init = inst.__init__
    except AttributeError:
	if args: raise TypeError, ('arguments given for class with no initialiser',
				   C, args)
    else:
	apply(init, (inst,) + args)

    return inst

#	Eddy.


From Edward Welbourne <eddy@chaos.org.uk>  Wed Dec  9 22:39:07 1998
From: Edward Welbourne <eddy@chaos.org.uk> (Edward Welbourne)
Date: Wed, 9 Dec 1998 22:39:07 +0000 (GMT)
Subject: [Types-sig] Re: a toy
Message-ID: <m0znsGB-0000FdC@vortigen.demon.co.uk>

# Doh !  Error in instance().

replace
	apply(init, (inst,) + args)
with
	apply(init, args)

Eddy.


From Edward Welbourne <eddyw@lsl.co.uk>  Thu Dec 10 18:38:05 1998
From: Edward Welbourne <eddyw@lsl.co.uk> (Edward Welbourne)
Date: Thu, 10 Dec 1998 18:38:05 GMT
Subject: [Types-sig] Meta-stuff
Message-ID: <199812101838.SAA20338@lsls4p.lsl.co.uk>

> ... One could easily be viewed as getattr behavior.  I like to
> think about it as the "Python view" on this discussion. 

Because python defines all its other behaviour in terms of attributes of
the object (see b, below), with those attributes implementing the
behaviour, *all* one ever needs is getattr behaviour.  Touch wood ;^}

> The other issue is dealing with the unification of classes and types.

I prefer to think of it as removing the artificial separation between
the two, rather than unifying them ;->

>> has somebody created a specific list of things to change in the
>> Python object model?

OK, here's a stab at that:

 i) Remove everything artificial from the python object model.

Which mostly comes down to

a) Remove, from attribute lookup, *all* references to the type of the
   object.

with a little bit of

b) Where a type provides functionality that `everyone else' has to
   provide by having the right method to define that functionality,
   expose that functionality (putatively as attributes of the type,
   accessed by objects of that type `in the same way' that an instance
   accesses those of its class) in such a way that (even if the python
   engine internally takes a short-cut) a python program *can access*
   the relevant methods in just the same way as it would if the object
   were an instance.  Thus I would sum a list of integers with
   reduce(Integer.__add__, sequence, 0), and get a type error if any
   entries in the sequence aren't integers.  (Well, maybe a coercion
   error, not sure - what does a type error become in the absence of
   types ?)


Does anyone else find the silences disconcerting ?

	Eddy.


From Edward Welbourne <eddyw@lsl.co.uk>  Fri Dec 11 10:34:35 1998
From: Edward Welbourne <eddyw@lsl.co.uk> (Edward Welbourne)
Date: Fri, 11 Dec 1998 10:34:35 GMT
Subject: [Types-sig] Meta-stuff
References: <199812101838.SAA20338@lsls4p.lsl.co.uk> <36703B97.200EF5C4@ixl.com>
Message-ID: <199812111034.KAA01140@lsls4p.lsl.co.uk>

> I must be misinterpreting this.  Without looking at the type, how can
> you discern the (underlying C/C++) structure of the object?  (and thus
> how can you do *anything* with it?)

I first met this problem 15 years ago and I concluded it couldn't be done.
I now know how to do it.

But I have to write it or no-one sane will believe it can be done.
And I have a full-time job keeping me from it all day today.
I may be a bit quiet 'til I'm done.
Maybe Monday.

	Eddy.


From Edward Welbourne <eddy@chaos.org.uk>  Sun Dec 13 22:20:20 1998
From: Edward Welbourne <eddy@chaos.org.uk> (Edward Welbourne)
Date: Sun, 13 Dec 1998 22:20:20 +0000 (GMT)
Subject: [Types-sig] Bleary
Message-ID: <m0zpJsC-00000OC@vortigen.demon.co.uk>

I remain fanatically convinced this is going to work.
I don't Know, but I do Believe.

I'm uploading a snapshot to http://www.chaos.org/~eddy/craft/tot.html
and links threfrom.  I shalln't claim it's ready to be read, but you are
welcome to the opportunity to work out for yourselves whether I've
finally gone off my trolley.  At least I haven't died unexpectedly ;^>

	Eddy.


From Edward Welbourne <eddyw@lsl.co.uk>  Mon Dec 14 13:06:55 1998
From: Edward Welbourne <eddyw@lsl.co.uk> (Edward Welbourne)
Date: Mon, 14 Dec 1998 13:06:55 GMT
Subject: [Types-sig] Re: Bleary
Message-ID: <199812141306.NAA19214@lsls4p.lsl.co.uk>

Oops - for chaos.org read chaos.org.uk, so the link should be to

http://www.chaos.org.uk/~eddy/craft/tot.html
		    ^^^

Sorry.

	Eddy.


From Edward Welbourne <eddyw@lsl.co.uk>  Wed Dec 16 17:48:43 1998
From: Edward Welbourne <eddyw@lsl.co.uk> (Edward Welbourne)
Date: Wed, 16 Dec 1998 17:48:43 GMT
Subject: [Types-sig] attempted summary
Message-ID: <199812161748.RAA03843@lsls4p.lsl.co.uk>

OK, so I'm getting a clear enough view of where I'm going that I can say
the things that seem to be relevant to where I started (which is
python's type system) rather than where I'm going (which is the
completion of A Quest that I was Given when I was an apprentice).

The tone of the types/classes/meta discussion was all pointing at the
fact that attribute lookup is the cornerstone: if we can just do that,
we can do everything (eg take the object you want to involve in
addition, look up its addition attribute, use that to do addition: all
we needed was attribute lookup).

Well, of course, we also need function invocation or we can't do that.

Now, it so happens one can formalise searching (eg attribute lookup) as
exception handling.  Roughly, this formalism looks at the code we use to
do attribute lookup and views it as *a piece of computation which is
meant to achieve nothing* but if you happen to find the attribute during
lookup, you raise the IveFoundIt exception, which the lookup wrapper
catches, and the rest of the futile computation gets skipped because an
exception was raised.  If the futile computation runs to completion
we're then (perhaps to the formalism's surprise) disapointed: we
probably raise IhaventFoundIt, which we're more used to viewing as an
exception.  I hope at least the perverts among you will enjoy that
thought - I certainly did when it caught my attention.

So I'm going to do attribute lookup as exception handling: furthermore,
the way I'm going about exception handling is to have a chain of
exception handlers *on the stack*, each pointing `up' to one further up
the stack than itself, right back to the ultimate handlers which are
static and hang off the top of the chain.  When an exception gets raised
it *immediately* takes a journey up the chain to find whoever is going
to catch it, negotiates with them and returns control to the context who
had the problem.  It passes enough information back to this context to
let it

either pretend none of that ever happened (exception resolved)
or know that something has gone wrong and return to its caller

In the latter case, the exception handling will have adjusted the
handlers on the stack so that they all tell their contexts to give up as
control returns to them, until it gets back to whoever caught the
exception, who will (presumably) do something about the mess.  I'm a bit
vague about what gets returned by the functions which are returning
here, but the crucial thing is that the exception handler can have
transcribed some data from the exception it handled to the handler (or
some record known to it).  So the value found by the lookup is `passed
back' to whoever asked for it *and then* the code which found it
abandons what it's doing, as if in response to an error.

Anyhows, we end up needing only function invocation and error handling.
For which I have a `good enough' very simple model which will let me
bolt everything in via the namespace lookup that the handlers can fake
up for me.  In this model, every `record' consists of a structure whose
first field is a *function* (in the implementation language) which is
known to other functions from the same *compilation unit* as it.  The
record may have other fields (in which case that compilation unit had
better know about this and be sure not to install the wrong function on
the wrong record type, because the *central system* knows nothing about
all this `cheating' that each type does), but the central system never
attends to them.  (Well, of course, the central system is a compilation
unit, too, so the records for which *it* is responsible may have such
extra fields as it cares for, so long as it puts the right functions on
them.)  The function serves as a `type' but only its compilation unit
*really* recognises it ... though other compilation units may own
lookups in which they can find tools with which to manipulate records
carrying that function.

Beyond that, the fiddly bit (that I fanatically Believe I Can Do but on
which I realise I might get stuck) is the building of a bunch of static
data structures, with functions hanging off them, which will constitute
(at compile time, in practice, though there will be some dotting of i
and crossing of t to do as a runtime initialisation) a lookup/exception
handling engine with enough machinery on board to let other modules:

 register interfaces, behaviours and (most important of all) some rather
 more sophisticated tools for doing lookup efficiently (but carefully
 disguised so the system never realises that's what they are).

 ask it for things registered with it, ie do lookup

and I'll be back to that on Friday, when I break from work 'til New Year.
I'm expecting that by the New Year I'll either have discovered what I've
failed to notice thus far, or have solved a problem that's bugged me for
13 and some years.  Manic but happy ;^>

My toy implementation's current fragments are at 
	http://www.chaos.org.uk/~eddy/craft/tot.html
but I'm afraid there are some typos in the links.  The second tot.c
should say zot.c, but fortunately its href is right; ankh.c is right but
its link says anhk.c so doesn't work; and I forgot to mention zot.h and
ankh.h, which are also there.  This page will evolve as I progress.

	Eddy.