Project organization and import

[Jorge Godoy]

"Martin Unsal" <martinunsal at gmail.com> writes:

> 1) Namespace. Python wants my namespace heirarchy to match my filesystem
> heirarchy. I find that a well organized filesystem heirarchy for a
> nontrivial project will be totally unwieldy as a namespace. I'm either
> forced to use long namespace prefixes, or I'm forced to use "from foo import
> *" and __all__, which has its own set of problems.

I find it nice.  You have the idea of where is something just from the import
and you don't have to search for it everywhere.  Isn't, e.g., Java like that?
(It's been so long since I last worried with Java that I don't remember if
this is mandatory or just a convention...)

You might get bitten with that when moving files from one OS to another,
specially if one of them disconsider the case and the other is strict with
it. 

> 1a) Module/class collision. I like to use the primary class in a file as the
> name of the file. However this can lead to namespace collisions between the
> module name and the class name. Also it means that I'm going to be stuck
> with the odious and wasteful syntax foo.foo everywhere, or forced to use
> "from foo import *".

Your classes should be CamelCased and start with an uppercase letter.  So
you'd have foo.Foo, being "foo" the package and "Foo" the class inside of it.

> 1b) The Pythonic way seems to be to put more stuff in one file, but I
> believe this is categorically the wrong thing to do in large projects.  The
> moment you have more than one developer along with a revision control
> system, you're going to want files to contain the smallest practical
> functional blocks. I feel pretty confident saying that "put more stuff in
> one file" is the wrong answer, even if it is the Pythonic answer.

Why?  RCS systems can merge changes.  A RCS system is not a substitute for
design or programmers communication.  You'll only have a problem if two people
change the same line of code and if they are doing that (and worse: doing that
often) then you have a bigger problem than just the contents of the file. 

Unit tests help being sure that one change doesn't break the project as a
whole and for a big project you're surely going to have a lot of those tests.

If one change breaks another, then there is a disagreement on the application
design and more communication is needed between developers or a better
documentation of the API they're implementing / using. 

> 2) Importing and reloading. I want to be able to reload changes without
> exiting the interpreter. This pretty much excludes "from foo import *",
> unless you resort to this sort of hack:
>
> http://www.python.org/search/hypermail/python-1993/0448.html
>
> Has anyone found a systematic way to solve the problem of reloading in an
> interactive interpreter when using "from foo import *"?

I don't reload...  When my investigative tests gets bigger I write a script
and run it with the interpreter.  It is easy since my text editor can call
Python on a buffer (I use Emacs). 

> I appreciate any advice I can get from the community.

This is just how I deal with it...  My bigger "project" has several modules
now each with its own namespace and package.  The API is very documented and
took the most work to get done.

Using setuptools, entrypoints, etc. helps a lot as well. 


The thing is that for big projects your design is the most important part.
Get it right and you won't have problems with namespaces and filenames.  If
you don't dedicate enough time on this task you'll find yourself in trouble
really soon. 

-- 
Jorge Godoy      <jgodoy at gmail.com>