Python module import loop issue

[Carl Banks]

On Dec 29, 10:51 am, Kottiyath <n.kottiy... at gmail.com> wrote:
> This might not be  pure python question. Sorry about that. I couldnt
> think of any other place to post the same.
> I am creating a _medium_complex_ application, and I am facing issues
> with creating the proper module structure.
> This is my first application and since this is a run-of-the-mill
> application, I hope someone would be able to help me.
>
> Base Module:
> Contains definitions for Class A1, Class A2
>
> Module 1.1:
> Class B1 (refines A1)
> Module 1.2:
> Class C1 (refines A1)
> Module 1.3:
> Class D1 (refines A1)
>
> Module 2.1:
> Class B2 (refines A2):
>         Uses objects of B1, C1, D1
> Module 2.2:
> Class C2 (refines A2)
> Module 2.3:
> Class D2 (refines A2)
>
> -->Python Entry Module : Module EN<--
> Calls objects of B1, C1 and D1
>
> Module EN and also Module 2 creates and calls the objects during run
> time - and so calls cannot be hardcoded.
> So, I want to use Factory methods to create everything.
>
> Module Factory:
> import 1.1,1.2,1.3,  2.1,2.2,2.3
> A1Factory: {'B1Tag':1.1.B1, 'C1Tag':1.2.C1, 'D1Tag':1.3.D1'}
> A2Factory: {'B2Tag':2.1.B2, 'C2Tag':2.2.C2, 'D2Tag':2.3.D2'}
>
> But, since Module requires objects of B1, C1 etc, it has to import
> Factory.
> Module 2.1:
> import Factory.
>
> Now, there is a import loop. How can we avoid this loop?
>
> The following ways I could think of
> 1. Automatic updation of factory inside superclass whenever a subclass
> is created. But, since there is no object created,  I cannot think of
> a way of doing this.

I'm going to suggest three ways: a straightforward, good-enough way; a
powerful, intelligent, badass way; and a sneaky way.


1. The straightforward, good-enough way

Define functions in Factory.py called register_A1_subclass and
register_A2_subclass, then call them whenever you create a new
subclass.

Factory.py
-----------------------------
A1Factory = {}
A2Factory = {}

def register_A1_subclass(tag,cls):
    A1Factory[tag] = cls

def register_A2_subclass(tag,cls):
    A2Factory[tag] = cls
-----------------------------

package1/module1.py:
-----------------------------
import Factory

class B1(A1):
    # define class B1 here

Factory.register_A1_subclass("B1Tag",B1)
-----------------------------

So after you define B1, call Factory.register_A1_subclass to add it to
the A1Factory.  Factory.py no longer has to import package1.module2,
so the circular import is broken, at the paltry price of having to add
a boilerplate function call after every class definition.


2. The powerful, intelligent, badass way

Metaclasses.  I would guess you do not want to do this, and I wouldn't
recommend it if you haven't studied up on how metaclasses work, but
it's a textbook example of their usefulness.  If you expect to use
factory functions like this a lot, it might be worth your while to
learn them.

Anyway, here's a simple example to illustrate.  It doesn't meet your
requirements since all classes use the same factory; updating it to
your needs is left as an exercise.

Factory.py:
-----------------------------
Factory = {}

class FactoryMetaclass(type):
    def __new__(metaclass,name,bases,dct):
        cls = type.__new__(metaclass,name,bases,dct)
        tag = dct.get("tag")
        if tag is not None:
            Factory[tag] = cls
        return cls
------------------------------

Base.py:
------------------------------
import Factory

class A2(object):
    __metaclass__ = FactoryMetaclass
    # define rest of A2
------------------------------

package1/module2.py:
------------------------------
class B2(A2):
    tag = "B2Tag"
    #define rest of B2
------------------------------

When the class B2 statement is executed, Python notes that the
metaclass for A2 was set to FactoryMetaclass (subclasses inherit the
metaclass), so it calls FactoryMetaclass's __new__ method to create
the class object.  The __new__ method checks to see if the class
defines a "tag" attribute, and if so, adds the class to the Factory
with that tag.  Voila.

(As a footnote, I will mention that I've created a library, Dice3DS,
that uses metaclass programming in exactly this way.)


3. The sneaky way

New-style classes maintain a list of all their subclasses, which you
can retrieve by calling the __subclassess__ class method.  You could
use this to define a factory function that searches through this list
for the appropriate subclass.

Factory.py:
-----------------------------
def _create_subclass(basecls,name):
    for cls in basecls.__subclasses__():
        if cls.__name__ == name:
            return cls()
        cls2 = _create_subclass(cls,name)
        if cls2 is not None:
            return cls2()
    return None

def create_A1_subclass(name):
    cls = _create_subclass(A1,name)
    if cls is None:
        raise ValueError("no subclass of A1 by that name")
    return cls
-----------------------------

So here you search through A1's subclasses for a class matching the
class's name.  Note that we do it recursively, in case B1 (for
instance) has its own subclasses, and I presume we do want those.

You can change it to do a search by a tag class attribute if you wish;
left as an exercise.


> 2. Update A1Factory in each module which implements refinements.
> _Very_important_, how do I make sure each module is hit - so that the
> factory is updated? The module EN will be looking only at base module,
> so the other modules is not hit. I will have to import every module in
> EN - just to make sure that the A1Factory updation code is hit. This
> looks in-elegent.

Not worth it.  The straightforward, good-enough way above is good
enough.


Carl Banks