creating many similar properties

[Michele Simionato]

Carl Banks wrote:
> You sound as if you're avoiding metaclasses just for the sake of
> avoiding them, which is just as bad as using them for the sake of using
> them.

Do you realize that you are effectively saying "avoiding a complex
tool in favor of a simpler one is just as bad as avoing the simple tool
in favor of the complex one" ?

> Here's how I see it: either it's ok to fiddle with the class dict, or
> it isn't.  If it's ok, then a metaclass is the way to do it.  If it's
> not ok to fiddle with the class dict, then he should be using
> __setattr__, or creating properties longhand.  Messing with frames is
> not the answer for production code.

I agree that messing with frames is not nice (however I should notice
that
this is how Zope interfaces are implemented, and they have been in
production use for years) but I disagree with your point about the
class dict. You should use a custom metaclass *only if you want to
mess with the class dict of all subclasses at each derivation*: this is

rarely the case, and definitely was not requested for the OP problem.

>
> Just for the hell of it, I decided to accept your challenge to run the
> standard library with a different metaclass applied to all new-style
> classes.  I ran the 2.4.3 regression test, replacing the builtin object
> with a class that used the mod256metaclass I presented in this thread.
> The results:
>
> 229 tests OK.
> 34 tests failed:
>     test___all__ test_asynchat test_cgi test_cookielib test_copy
>     test_copy_reg test_cpickle test_decimal test_descr test_descrtut
>     test_email test_email_codecs test_httplib test_imaplib
>     test_inspect test_logging test_mailbox test_mimetools
>     test_mimetypes test_minidom test_pickle test_pyclbr
>     test_robotparser test_sax test_sets test_socket test_socket_ssl
>     test_sundry test_timeout test_urllib test_urllib2 test_urllib2net
>     test_urllibnet test_xpickle

34 tests failed, worse than I expected.

> Not A-OK, but not exactly mass-pandemonium either.  There were plenty
> of modules used the the new base object and worked fine.
>
> There were only two causes for failure:
> 1. A class attempting to use __weakref__ slot.
> 2. There were also a few metaclass conflicts.

Yes, this agree with my findings. I was curious to know if there were
additional issues.

> IMO, neither of these failure modes argues against using a metaclass to
> preprocess the class dict.

But they argue against using metaclasses in general, IMO! (or at least,
against using them for users that are not aware of all the potential
pittfalls).

> The __weakref__ error is not applicable;
> since it's an error to use it on any class with an instance dict.  (In
> fact, the metaclass wasn't even causing the error: the same error would
> have occurred if I had replaced builtin object with an empty subclass
> of itself, without the metaclass.)

Correct, this more of a problems of __slots__ that play havoc with
inheritance
than a problem of metaclasses.

> The metaclass conflict would only occur in the present case only if
> someone wanted to subclass it AND specify a different metaclass.
> Arguing that metaclasses should be avoided just to guard against this
> rare possibility is defensive to the extreme.

Not too extreme in my opinion. Real life example: I had a debugging
tool
using a custom metaclass, I tried to run it on Zope 2.7 classes and I
have got segmentation faults. In Zope 2.8 I get "only" metatype
conflicts, and to avoid that I had to rewrite the tool :-(

> I did not uncover any kinds of subtle, unexpected behavior that can
> occur when metaclasses do weird things.  I know such things are
> possible; how likely they are is another question.  The tests I ran
> didn't uncover any.  So for now, the results of these tests don't seem
> to support your point very well.

Well, this is a matter of opinion. In my opinion your tests support my
point pretty well, better than I expected ;)

>
> Appendix: Here's how I ran the test.  I inserted the following code at
> the top of Lib/test/regrtest.py, and
> ran make test.  I ran the tests on the Python 2.4.3 source tree, in
> Linux.
>
> =======================
> import sys
>
> class mod256metatype(type):
>     def __new__(metatype,name,bases,clsdict):
>         print >> sys.__stdout__, \
>               "++++++++ Creating class %s of type mod256metatype" %
> name
>         def makeprop(sym):
>             prop = '_%s'  % sym
>             def _set(self,v):
>                 setattr(self,prop,v%256)
>             def _get(self):
>                 return getattr(self,prop)
>             return property(_get,_set)
>         for sym in clsdict.get('__mod256__',()):
>             clsdict[sym] = makeprop(sym)
>         return super(metatype
>         return type.__new__(metatype,name,bases,clsdict)
>
> class _object:
>     __metaclass__ = mod256metatype
>
> import __builtin__
> __builtin__.object = _object
> =======================

I used a similar approach. I added in sitecustomize.py the following
lines:


import __builtin__

class chatty_creation(type):
    "Print a message every time a class is created"
    def __new__(mcl, name, bases, dic):
        try:
            cls = super(chatty_creation, mcl).__new__(mcl, name, bases,
dic)
        except Exception, e:
            print e
            print 'Could not enhance class %s' % name
            cls = type(name, tuple(b for b in bases if b is not
Object), dic)
            # removing Object from the bases is enough only in the
trivial
            # cases :-(
        else:
            print 'Creating class %s.%s' % (dic.get('__module__'),
name)
        return cls

class Object:
    __metaclass__ = chatty_creation

__builtin__.object = Object

Now it is impossible to run both Zope and Twisted due to metatype
conflicts.
I haven't looked in detail, but the first conflict in Twisted is due
to a  metaclass-enhanced class which is setting properties. This is
the typical example of what I call metaclass *abuse* in my paper, since
the custom metaclass could have been avoided (for instance using George
Sakkis trick) and the conflict could have been avoided.

Now, I know how to solve the conflict
(http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/204197) but I
would rather avoid it altogether.

The problem with metaclasses is that you are adding magic to the
classes of
your USERS, and the users are known to play any kind of dirty tricks.
You
(speaking in general of you as the author of a framework) should strive
to keep things clean as much as possible.

I agree that the problems are rare: but just for this reason they are
prone to very subtle bugs, the hardest to find. And there is no
documentation of metaclass pittfall AFAIK :-(


 Michele Simionato