Best Practices for Internal Package Structure

[Steven D'Aprano]

On Wed, 6 Apr 2016 09:54 am, Ethan Furman wrote:

> On 04/05/2016 04:38 PM, Steven D'Aprano wrote:
>> On Wed, 6 Apr 2016 04:40 am, Ethan Furman wrote:
>>
>>> Well, there should be one more module:
>>>
>>>     test.py
>>>
>>> So in total, two files
>>>
>>> bidict/
>>> |-- __init__.py
>>> |-- test.py
>>
>>
>> Your test code shouldn't necessarily be part of the package though. If I
>> already have a package, then I will usually stick the test code inside
>> it, but if I have a single module, I keep the test code in a separate
>> file and don't bother installing it. It's there in the source repo for
>> those who want it.
>>
>>
>>> will do the trick.  Oh, and you want a README, LICENSE, a doc file.  And
>>> that should do it.  :)
>>
>> None of which ought to be part of the package itself. Well, perhaps the
>> README.
> 
> If it's not part of the package, how does it get installed?  And where?
>   (Honest question -- I find the packaging and distribution process
> confusing at best.)

Why do they need to be installed? They're part of the development
environment, not the module.

(Actually, I take it back about the README, since conventionally that's used
to give installation instructions. Once the module is installed, you don't
usually need the README any more. But the doc file might be more useful.)


I don't know what life is like in the brave new world of wheels, eggs, pip
and other new-fangled complications^W tools, but in the good old fashioned
setuptools world, you have something like a zip file or tarball that
contains your code, plus an installation script. The contents of the zip
file will be something like this, taken from PyParsing version 2.1.1:


build/
CHANGES
dist/
docs/
examples/
htmldoc/
HowToUsePyparsing.html
LICENSE
MANIFEST.in
PKG-INFO
pyparsingClassDiagram.JPG
pyparsingClassDiagram.PNG
README
pyparsing.egg-info/
pyparsing.py
robots.txt
setup.cfg
setup.py

You unzip the file, cd into that directory and run:

python setup.py install

which then magically installs the module somewhere that just works. In this
specific case, it installs the module in a file

/usr/local/lib/python3.3/site-packages/pyparsing-2.1.1-py3.3.egg

which happens to be a vanilla zip file containing the pyparsing.py file, a
byte-code compiled .pyc version, and a directory EGG-INFO that presumably
allows pip or whatever to uninstall it again, or something, who the hell
knows how this works.

But the installer could just have easily have just installed the
pyparsing.py file alone.

The point is, the tests, documentation, etc aren't necessarily part of the
module, so they don't necessarily need to be installed into your
site-packages. If the user wants to read the HTML docs, they can browse
into the expanded source tarball and read them there, or drag the docs into
their home directory, or whatever they feel like. If they want to run the
tests, they run them from the source directory, not site-packages.

Of course, if you prefer to keep the tests inside the package, that's up to
you. But you needn't feel that this is the best or only place for them.



-- 
Steven