[Python-Dev] On breaking modules into packages

[Kristján Valur Jónsson]

Just a small input into this discussion:

In EVE, for historical reasons, we implemented our own importing mechanism.  I think it is because we started out with an ancient Python version that didn't support packages.
Regardless, we still have a system where a hierarchy of files is scanned, and then code in each .py files determines where in the "namespace" it lands.  This can be
Declaratively (by using a __guid__ attribute on a class, for instance) or by defining a special __exports__ dict at the module level.
The good thing about this system is that it allows us to separate code in a manner independent of the api.  We can choose for example to group all network
Code in a folder.  Or have each class in the "game.entity" namespace be defined in its own file.  It unhooks file structure from name structure.

Now, this has its own problems of course, the biggest of it being that it is non-standard.  Off the shelf IDEs have problems with it.  And we have to implement dynamic reloading on our own.  The list goes on, and for that reason, we are moving away from it in favor of standard python import.

However, I am personally not super happy about how this will force one to think in "api" terms when creating source files.  As has been mentioned, files cannot be moved and restructured once in general use, and when writing new code, one has to think long and hard about "where" to put the source, not "what" to put in it.  What is more, a hierarchy, while a convenient system for storing files, does not, IMHO, always map to problem domain.

But we're having a go at it.
Time will tell if "forcing us to think inside the hierarchy" will be beneficial in the long run.

Cheers,
K