python project layout

[Peter Hansen]

Huron wrote:
>         What do you guys recommend in terms of python project layout, especially
> unit tests layout ?
>         Zope has unit tests per packages, twisted has a big tests directory full of
> tests ... and the file naming convention are also pretty differents ...
>         I guess there is part of "personnal choices" in those matters ... but there
> is also good practice advices that can be gathered I think ...

(This is the result of a few years of evolution of our source tree 
conventions.)  We place all tests in subfolders (named "tests") of the 
folders containing the files under test.  Unit tests are named for the 
unit (module) they are testing, so serial.py would have a unit test file 
at tests/serial_unit.py.

"Acceptance" tests (higher level usually black-box functional tests) are 
in the "tests" folder of the top level source folder, along with any 
unit tests that reside there, and take their name from the number of the 
requirement that defined them.  Since we're an XP team, we call these 
requirements "stories", so the files are named exciting things like 
"story001.py" and "story197.py".  The specific numbers match up with 
either index cards with descriptions of the requirements, or an online 
"issue tracker", depending on the project.

We have a little (quite simple) internal utility that lets us easily run 
all unit tests under the current folder, or all acceptance tests in the 
project, with one command.  (Specifically, "uix" and "stix" for unit and 
story tests, respectively.)  This "test runner" utility scans subfolders 
(doing an os.walk basically) and executes each test script in a new 
process, to avoid most potential undesirable interactions between tests.

The benefits are straightforward.  Tests are easy to find for any file, 
but don't "clutter up" the same folder as the files under test.  The 
naming conventions makes it easy to automate things like the test runner 
or other utilities.  Running tests (always from a console) takes only a 
few seconds so we can do it often.  Multiple projects are easily 
supported in other directory trees.

-Peter