caught in the import web again

[Steven D'Aprano]

Charles T. Smith wrote:

> Yes, we're talking about recursive imports here.  It's a complex, object-
> oriented system with big classes and little classes that are strongly
> interrelated.

Well, there's your problem right there. You're working with a complex,
highly coupled code-base. Alarms bells should be ringing in your head. That
sort of highly-interdependent code base is a nightmare to maintain at the
best of times. Recursive imports are just the start of the problem.

Circular dependencies are not just a problem in Python, they are a problem
throughout most of software design.

https://en.wikipedia.org/wiki/Acyclic_dependencies_principle

http://stackoverflow.com/questions/1897537/why-are-circular-references-considered-harmful


> I can get the imports configured properly so everything 
> works but if I make a little change to the code, then suddenly all my
> imports are broken and I must painstakingly go through the whole
> structure, reorganizing them.

The problems you are having with recursive imports are the symptom of a
deeper design problem. If you can, you should try to refactor your code so
as to have fewer mutual dependencies and less coupling.

 
> Are others equally frustrated by this or is there a trick or principle
> that I'm missing.  

Probably, and yes. Personally, I try very hard to avoid recursive imports as
a matter of principle, even when I can manage them safely, so personally
I'm never frustrated by them. But there are a few tricks you can use to
reduce the pain a little. Think of these as pain-killers. All they are
doing is masking the symptoms of an underlying problem, namely excessive
coupling.

You can often delay imports to when a function or method is called. Although
this is not "best practice", it does have its uses. Instead of this
recursive import:

# spam relies on this module, but this module relies on spam
import spam
class K:
    def __init__(self):
        self.eggs = spam.stuff()


you can write this:

class K:
    def __init__(self):
        import spam
        self.eggs = spam.stuff()


That delays importing spam until after this module is already fully
initialised. If you're worried about efficiency, don't be, only the very
first import of spam will be expensive, the others will be very fast.

Another way to avoid recursive imports is to merge the code from both
modules into a single module. Note that this is the complete opposite of
the conclusion you draw below.

Google for "avoiding circular imports python" (and similar) and you will
find lots of discussions about this. Leave out the "python" and you will
find the same thing for other languages.


> At this point, I guess the way I'll have to proceed is 
> to put every class in its own file, no matter how small.  Hopefully that
> takes care of the problem.

I don't think that's going to help. If anything, it will make it worse.
Suppose you have two classes, A and B, which require each other:

class A:
    def __init__(self):
        self.thing = B()

class B(A): pass


There's a circular dependency right there, but no import is needed so it is
comparatively easy to manage. Now move them into separate modules, and
you've turned a manageable circular dependency within a single module into
a painful recursive import.

Python is not Java, nor Perl, and if you're putting every class into its own
file, you are doing it wrong.


> I compared perl's object-oriented philosophy with python's and didn't
> like how perl *requires* you to put everything into its own file.  Now it
> looks like python does, too, implicitly.

You are mistaken about that.



-- 
Steven