What is the status of the __subclasses__ method?

[Michael Hudson]

Ruud de Jong <ruud.de.jong at consunet.nl> writes:

> Michael Hudson schreef:
> 
> > Ruud de Jong <ruud.de.jong at consunet.nl> writes:
> > 
> >>The question I have is: how safe / future proof / portable is the
> >>use of the __subclasses__ method that exists for new-style classes?
> > Hmm.  I think it's unlikely to go away.
> > 
> >>When I searched python.org for "__subclasses__" I found
> >>some more information. The __subclasses__ method appears
> >>to exist to allow modifications of the superclass to be
> >>percolated down to its children, mainly for speed reasons,
> >>if I understand Tim Peter's explanation correctly
> >> (http://mail.python.org/pipermail/python-list/2003-August/176360.html). 
> > Yep.
> 
> What I wanted to avoid is that I would rely on something that is
> not part of the Python *language*, but rather of a Python
> *implementation*. But then, the distinction between these two
> is not always clear

Indeed.

> -- if a construct is present in every implementation, what is the
> difference with it being a part of the language?
> 
> Well, raising the question is answering it -- as long as a
> construct is not officially part of the language, that construct
> can in principle change or disappear if the language maintainers
> find that necessary.

I'm not sure this attitude really applies to Python.  There's no real
definition of what "officially part of the language" means.

We don't take things away for the fun of it.  __subclasses__() would
only disappear if some major internal restructuring happened and it
becamse massively inconvenient to keep it.  But this applies (probably
most of the time with less force) to just about anything else!

> >>I also found several warnings that the __subclasses__ method only
> >>returns classes that have been accessed previously (which would
> >>defeat my purpose)
> > Hum, you're being overly paranoid here.  Python defined types are
> > PyType_Ready-ed as soon as they are created.
> > 
> 
> Well, I guess that such cautious behavior has become sort of
> second nature for me -- a consequence of having worked some
> twenty years on a huge software system (think multi-million lines
> of C code), with thousands of colleagues adding, deleting and
> modifying code simultaneously. In spite of strict processes and
> procedures, too often somebody would make themselves dependent on
> some construct in another part of the system that was never
> meant for public usage. 

It's exposed to Python.  It it was really meant to be internal, that
wouldn't have happened.

> Anyway, if I understand you correctly, as long as the subclasses
> are implemented in Python, they will be known to their parent
> immediately.

Yes.

> And from your other remarks I get the impression that the
> __subclasses__ method is a rather permanent feature in every Python
> implementation.

Well, it's only present in one implementation at the moment, for what
that's worth...

> > 
> >>So again, my question. Is it safe to use the __subclasses__
> >> method for the purpose I described above? 
> > Well, I can't see into the future, but I'd feel secure using it.
> > I'm not sure what you describe is a very tasteful use of it
> > though...
> > 
> Well, not being a native english speaker, I don't quite know how to
> interpret this latter remark. Please elaborate. Are you objecting
> to the use of the __subclasses__ method?

No.

> Or to the general mechanism I described?

I didn't really spend long trying to understand what you said you were
trying to do, but it sounded a little strange.  That's all.

> Here is the essence of the use of __subclasses__ in the application
> I referred to. Note that is a variant of the Chain-of-Responsibility
> pattern, applied to the Factory Method. I know that I abused the
> NotImplementedError -- in the actual application it is a custom-
> defined exception. But for illustration purposes I wanted to avoid
> such non-essential details.
> 
>  >>> class Event(object):
> 	def __new__(cls, line):
> 		for subclass in cls.__subclasses__():
> 			try:
> 				return subclass.__new__(subclass, line)
> 			except NotImplementedError:
> 				continue
> 		raise NotImplementedError
> 
> 	
>  >>> class MsgEvent(Event):
> 	pass
> 
>  >>> class IncomingMsgEvent(MsgEvent):
> 	def __new__(cls, line):
> 		if line.startswith('RCV:'):
> 			return object.__new__(cls)
> 		raise NotImplementedError
> 
> 	
>  >>> class OutgoingMsgEvent(MsgEvent):
> 	def __new__(cls, line):
> 		if line.startswith('SND:'):
> 			return object.__new__(cls)
> 		raise NotImplementedError
> 
> 	
>  >>> x = Event('RCV: blah blah')
>  >>> x
> <__main__.IncomingMsgEvent object at 0x00AD2D30>
>  >>>

That's cute.

> The thing to note is that I can add subclasses at will, and never
> have to revisit the Event root class. This means better
> maintainability and extensibility.

Yes.  I've actually used __subclasses__ for a somewhat similar
purpose, having a Resource class whose subclasses know how to find all
instances of each resource type and so being able to find all
resources by looking through subclasses.

I will note that there's some thing *slightly* odd going on here,
which perhaps is highlighted by considering what happens if you
subclass OutgoingMsgEvent.  In the code you posted, this subclass
wouldn't be picked up.  You could traverse the subclass graph easily
enough, but it seems to me that "is able to instantiate events" is
more like a "class-instance" relationship (flat) than a
"class-subclass" relationship (possibly nested) and so it might be
more appropriate to make all your Event classes instances of a custom
metaclass that keeps track of its instances and then create events by
calling a method on the metaclass.

But, whatever.

> > 
> >>Or would it be safer to revert to my home-grown metaclass solution?
> > You might want to keep it around, I guess...
> > 
> I will. I joins my collection of APL one-liners and my FORTRAN V
> preprocessor :-)

:-)

Cheers,
mwh

-- 
  Linux: Horse. Like a wild horse, fun to ride. Also prone to
  throwing you and stamping you into the ground because it doesn't
  like your socks.         -- Jim's pedigree of operating systems, asr