assignment to __class__ (was Re: Copy constructors)

[Glyph Lefkowitz]

On Sun, 12 Aug 2001, Guido van Rossum wrote:

> Glyph Lefkowitz <glyph at twistedmatrix.com> writes:
> 
> > Am I correct in understanding from this thread that there is an intent to
> > remove the ability to assign an instance's __class__ attribute?
> 
> Yes, I'd like to remove this.  See my previous post in this thread for
> more of an explanation of the problem.  But I haven't decided yet!
> This thread will help me figure out how big of a deal it will be.

It's a pretty big deal for me, but it sounds like we should be able to
reach some agreement, as the exact semantics of it aren't too important to
me, rather the loss of the features it provides. (see below for a
suggestion on how to do the check...)

> Also note that nothing will change (yet) for classic classes -- in
> 2.2, classic classes will use a different metaclass from "new-style"
> classes, and the classic metaclass will provide the same semantics for
> classes and instance as before.  The class statement creates a classic
> class by default -- unless you explicitly subclass from a built-in
> type or a new-style class.

Can you point me (and other readers just coming to this discussion) to a
few URLs illuminating the key differences between 'new-style' and
'old-style' classes?  I'm going to do some reading up on the various PEPs,
but if there are any posts on python-dev I can refer to...

> In a sense, in 2.2 the new-style classes will still be experimental,
> and it's quite likely that based upon feedback from users they will
> change (for the better) in later versions.

Good to know.  I have to say that although I am among the biggest
detractors to change, I can appreciate the difficulty of what you're
doing; python is the first language I know of that has ever gone through
significant *refactoring* (not whole-scale rewriting or just adding
things) at both the implementation and design level.  I wish you good
luck, and I hope that the voices from the "loyal opposition" are more of a
help than a hindrance.

[ (snip) so much is changing ]

> But you're getting so much in return!

> Subclassing built-in types,

Operator overloading got me 90% of the way there, and that was really the
only 90% I care about.  aside from isinstance() working on instances that
are 'like' integers now, what have I gained?

> get/set methods,

I already *have* get/set methods, in 1.5.2; see
twisted.python.reflect.Accessor :-)

> class and static methods,

Those could be easily faked before, for the OO zealots; but I actually
*like* the idea of using functions for things like that.  I prefer to
organize my code at the module level, and I find the additional option of
these method types just clutter.

> uniform introspection...

Aye, now there's the rub.  If we have uniform introspection, there's a
certain expectation that features like this become _easier_ to use, not
harder.  Introspection is a powerful feature, all the more powerful if
it's uniform and systematic.

[ (snip) promise pattern is easy with class assignment ]

> If you know the type it's going to be eventually, you can use
> C.__new__() to create an uninitialized C instance.

The point is that sometimes you don't...

Of course, you *could* create a class of which all your 'promised' objects
are instances, and do specialization by having a reference in each
instance to its 'real' class, but doesn't that seem a little silly given
that we don't have to do it now? :-)

> I guess the name ("twisted") says it all. :-)

Just wait until you get into the subproject naming scheme :)

> > One of the eye-popping cool features of Python is the ability to change code
> > and have existing instances update to use the new methods automatically.
> 
> You will still be able to modify *classes* dynamically -- although you
> have to declare this option by putting __dynamic__ = 1 in your class
> statement.

Hmm.  This seems like _less_ uniform introspection to me.  I do have to
note that if this is a requirement, then a part of the Twisted coding
standard will be to have all classes have __dynamic__ = 1; one rarely
knows what code is going to have a bug _before_ the server is started :)

Also, as an aside: one of my favorite things about Python is the ability
to fix bugs in a library you're using without having to modify the source
to that library (if libmodule.version == '0.6.0': fix_libmodule_bug()).  
This greatly eases deployment.  Fixing bugs in a running server is also
pretty important if you don't have the option to take the server down...

Could we have the default be the other way 'round?  (Has there already
been a discussion of that?)

[ Object>>become: ]
> Probably not -- although I don't know what that does.

Since smalltalk objects are effectively handles, you can replace all
references to one object with references to another.  I really don't
expect python to ever be able to do this.  (Although currently it can fake
it _really_ convincingly -- everything but 'is', pretty much --
instance-to-instance, by replacing __dict__ and __class__)

[ (snip) Can't you write a check to validate __class__ assignment? ]

> I'd rather not because it's a complicated check to write, and it may
> be difficult to explain the restrictions.  Here's an example of the
> kind of restriction that is unavoidable.
> 
> I don't know how familiar you are with Python's C-level internals.  
> If you are, you'll appreciate the problem if I took a list object and
> changed its type pointer to the dictionary type -- the instance
> lay-out of a dictionary is different, and all the methods would be
> using the list data as if it were dictionary data.  Recipe for
> disaster.  Likewise, changing a featureless object into a list or dict
> would at the very least require growing the size of the instance; this
> would require a realloc(), which may move the object in memory.  But
> if there are other references to the object, these would all have to
> be updated.  Python's run-time architecture just doesn't support that.

Yes.  At least, I *think* I'm familiar with that.

The obvious solution, if I understand python's internals correctly, is to
make a type method slot, "tp_change_class".  Most types would just raise
an exception; instances would continue to work the way they have been.  
Aside from the fact that it's icky syntactically, is there anything that
this would break or make unpleasant?

And if the syntax is not dealable, it would be a perfectly reasonable
transition to have

 def change_class(obj, newclass):
   obj.__class__ = newclass

be the current implementation of a function that would be implemented as a
builtin in the future...

> Nevertheless, all evidence suggests that Twisted is not typical Python
> code. :-)

It may not be typical, but I feel that the introspection features that
Twisted makes use of should not be considered hacks or kludges; they are
_very_ powerful tools.  I realize that some of them are the result of
implementation tricks but that does not make them any less useful,
powerful, or profound.  (More on this below.)

> Python is more dynamic than the language I *wanted* to design.

Thank you for screwing up, guido! :-D

Seriously, the dynamic nature of Python is what makes it cool.  I can
understand removing meaningless dynamic features in order to make it
faster (write access to locals() as a dictionary, for example) but I
suspect that many more dynamic features than you like are really really
useful.

If you want to design a better language in this regard, it may be
worthwhile to consider writing a language specifically designed for
producing more static python-like code, that uses the Python runtime to
present an external interface; (each module 'compiles' to a python-C API
file) especially if there are efficiency concerns.  I'd *really* like a
way to push my code to a "lower level" without resorting to C; such a
language ("Cleese" maybe?) having _some_ dynamic characteristics of python
but lacking others would be fine.

This makes the 'parrot' proposal sound even better; if the more worrisome
developments of python could take place in a radical fork (Py3K), but
remain compatible on a module-binding level with existing Python...

I could imagine an effort to make Python less dynamic could well end up
like the ill-fated project to make C more dynamic (C++).

> Some of the dynamicism was simply a implementation trick.  Some of the
> dynamicism is getting in the way of optimizing code, because the
> optimizer can never prove that certain variables won't be changed.

I want my Python code to execute faster, sure.  But before you start
eliminating features for the sake of speed, ask yourself -- is anyone who
is really concerned with efficiency writing code in *python*?  The speed
freaks have long since moved over to ADA or C++ or some other similiarly
torturous language to hate themselves at the speed of light while we're
having fun slowly. :-)

> So I'm trying to look for ways that pin down things a bit more.  I'm
> making assumptions about how "typical" Python code uses the dynamic
> features, and I'm slowly trying to introduce restrictions in the
> language that make the optimizer's life easier without affecting
> "typical" code.

Allow me to be skeptical of the fact that there is such a thing as
"typical" code :).  Most 'typical' python code could probably be written
in Java without much difference except a little more typing. It's when you
get to the boundary conditions -- adding attributes dynamically,
reflection, reloading, deploying, porting -- that's where Python starts to
shine.

Twisted is code which will probably only be written once, at least by me.  
It _should_ only be written once.  However, just because I hide the call
to socket.setblocking(0); socket.recv() behind a wrapper, and only call it
once (one line out of tens of thousands -- *highly* atypical!) does not
mean I'd be happy if nonblocking I/O went away.  Similiarly, if you remove
the ability to dynamically reconstruct a module (and patch up existing
instances of classes in that module in at most O(k*n) time, even if
there's a big k), there is only maybe 30 or 40 lines of my code which will
be affected; but it will change my entire development style.  It would
make me significantly less happy with python.

If only 1 function in my entire application has to be written in C, but I
can take advantage of the super-dynamic nature of Python elsewhere -- that
does not mean that it is a proportionally less important feature of python
that it can call C code.

And finally, at the risk of beating a dead horse to its second demise, as
I once heard a wise man say, "It is bad to use eval; it is worse not to
have it."

>From my perspective, usually the least typical code is the most important.

> For example, we're looking into optimizing access to builtins.  For
> this, we need to assume that the __builtin__ module is immutable; in
> addition, if a module doesn't have a global 'len', for example, we
> have to assume that such a global won't be inserted into the module
> dynamically.  I'm only aware of a very small number of applications
> that violate this constraint; I'd rather provide a separate explicit
> mechanism to override built-in functions so that the optimizer can be
> aware of a potential change and avoid it.

You won't get an argument from me there, at least; __builtins__ always
struck me as weird anyway ^_^.

but-making-overridingly-global-variables-is-not-quite-the-same-as-
	making-your-code-dynamic-and-introspective-ly-y'rs,

                      ______      __   __  _____  _     _
                     |  ____ |      \_/   |_____] |_____|
                     |_____| |_____  |    |       |     |
                     @ t w i s t e d m a t r i x  . c o m
                     http://twistedmatrix.com/users/glyph