Changing base class of a big hierarchy

[Alex Martelli]

<posted & mailed>

John J. Lee wrote:

> I'm trying to change a base class of a big class hierarchy.  The
> hierarchy in question is 4DOM (from PyXML).  4DOM has an FtNode class
> that defines __getattr__ and __setattr__ that I need to override.
   ...
> Obviously, I don't want to manually derive from every damned leaf
> class.  I want to say "give me a new class hierarchy just like this
> one, but derived from BrowserFtNode instead of FtNode".  I also don't
> want to actually mutate the xml.dom module, of course because other
> code may be using it.

OK, so you want to do two things:
  -- "deep-copy" a class hierarchy H to H' 
  -- in the copied hierarchy H', change each occurrence in any __bases__
     of a class X' to a class Y', and any other class T that is in H to
     the corresponding class T' in H'

> What's the easiest way to do that?

I think performing these two steps in order is probably simplest.  You
do have to identify every class in your starting hierarchy H, of course.

> Maybe deep-copying the module or something, then fiddling with
> __bases__??  Or a metaclass?

Deep copying might not help -- copy.deepcopy(X), when X is a class,
still leaves __bases__ unchanged.  A custom metaclass that does
bases-alteration as you require would be reasonably easy to write,
but would be more helpful than just doing the same job in a
function if, and only if, you were to re-execute every affected
'class' statement using the new metaclass.  I doubt that getting
and re-executing those class statements is simplest in this case.


> While I'm on the subject, does anybody have code to get a list of all
> classes in a package that derive from a particular class?  Or
> something I could use to do that easily?

I'm not sure of how to identify "all classes in a package" in the
simplest way.  All classes in a module, or other single container,
is of course easy (and actually you only want classes that derive
from X, so you'd filter that with issubclass); but offhand I can't
think of a handy, elegant and robust way to ensure you're walking
all over a package (including sub-packages), nor even that all of
a package (cum subpackages) is actually LOADED at this time.  I
suspect that for this purpose you will need to perform file-system
checks about what files in a package-directory are python modules,
what directories are sub-packages, and so on (I'd love to be
wrong on this point... maybe I just haven't given it enough thought...
for all submodules within the packages that ARE loaded, looping
over sys.modules.keys() and checking for the appropriate prefix
would do, but HOW else would one ensure that every single subpackage
and module therein is already loaded, save by filesystem ops...?).

If you can get all the modules of interest, say by an iterator,
then getting all classes of interest is reasonably easy.  Altering
them as you require is not all that hard, except that the ORDERING
in which you're looking at the class objects might be wrong -- it
would be simplest if you could be ensured of only looking at a
class after you've looked at all of its ancestors.  I think we can
do that by playing with the (inevitable) dict of class translations.
Something like:

# seed this with the base class transformation
trans_dict = {X: transformed_X}
# recursive way to transform a class between hierarchies
def transform_class(C):
    if not issubclass(C, X):
        return C
    if C in trans_dict:
        return trans_dict(C)
    bases = tuple([transform(B) for B in C.__bases__])
    newclass = type(C)('C', bases, C.__dict__)
    trans_dict[C] = newclass
    return newclass

Of course, if the classes in the hierarchy use each other in
more ways than just a base/subclass inheritance [e.g, some
classes have other classes as attributes, not just as bases]
you may need a bit more work [e.g., on C.__dict__ as well as
on C.__bases__].  But I think this is probably the core of
what you want, to be applied to a way to loop over all the
classes that may interest you, and store the transformed
classes somewhere -- that 'somewhere' of course will need to
be where some code that uses these classes (and originally
written to use classes from the original hierarchy) will look
to get the class-objects it uses.  You may need to alter the
semantics of 'import' statements in that using-code, too, since
you did say you do NOT want to alter the actual packages and
modules that contain the untransformed [original] classes,
and import statements for those modules that you do not
want to alter is probably how existing using-code gets its
access to those classes... but this is beyond the issue of
how to MAKE the new slighty altered hierarchy, and into the
separate issue of how to get some existing code to USE the
new altered hierarchy once you've made it.


Alex