How to fix my imports/file structure

[Terry Reedy]

On 1/20/2016 8:26 PM, Travis Griggs wrote:
> I wrote a simple set of python3 files for emulating a small set of mongodb features on a 32 bit platform. I fired up PyCharm and put together a directory that looked like:
>
> minu/
>      client.py
>      database.py
>      collection.py
>      test_client.py
>      test_database.py
>      test_client.py
>
> My imports are simple. For example, client.py has the following at the top:
>
>      from collection import Collection
>
> Basically, client has a Client class, collection has a Collection class, and database has a Database class. Not too tough.
>
> As long as I cd into the minu directory, I can fire up a python3 interpreter and do things like:
>
>      >>> from client import Client
>      >>> c = Client(pathstring='something’)
>
> And everything just works. I can run the test_files as well, which use the same sorts of imports.
>
> I'd like to modularize this, so I can use it another project by just dropping the minu directory alongside my application's .py files and just have everything work. E.g.
>
>      SomeDirectory/
>          application.py
>          minu/
>              …
>
> and application.py does something like:
>
>      from minu.client import Client
>
> When I try this though, and am running python3 from another directory, the local imports don't work. I placed an empty init.py in the minu directory. That made it so I could import minu. But the others broke. I tried using things like
>
>      from .collection import Collection #added the dot
>
> but then I can't run things in the original directory anymore, like I could before. What is the simple/right way to do this?
>
> I have looked around a bit with Dr. Google, but none of the examples really clarify this well (at least, for me), feel free to point out the one I missed.

Summary: there are two solutions, one of which I think is better.

Imports of python-coded files are done by searching the directories, 
usually not packages, on sys.path.  Python prepends '' to sys.path, 
standing for the 'directory that contain the startup file'.  In the 
first case, startup is client.py and '' is 'minu', so other modules in 
minu can be imported directly.  In the second case, startup is 
application.py and '' is SomeDirectory, so other modules in 
SomeDirectory can be imported.  There is only one other module, minu. 
The modules within minu must be imported via Minu.

What you want to do is make the two situations identical with respect to 
minu module imports, so the import code will work the same in the two 
scenarios.

1. Make scenario 2 match scenario 1.  In application.py, add 
sys.path.insert(1,path/to/minu).  Then import minu modules directly.

2. Make scenario 1 match scenario 2 in the sense of having the directory 
containing minu be on sys.path.  But be clever.   Suppose minu lives in 
directory Projects.  In your 3.x Lib/site-packages directory, add file 
'Projects.pth' containing path/to/minu (without quotes!).  This makes 
Projects an extension of site-packages, and inserts minu 'virtually' 
into site-packages, so indirect imports for any user.

The negative is having to do this for every 3.x version you want to use 
minu with.  But new versions only appear every 18-24 months, and copying 
the file into a new site-packages is not a big deal.

The positive is that you no longer need to drop minu into SomeDirectory 
but can import its modules into any application module as if minu were 
in site-packages, and as it would (should) be if you were to distribute 
minu to others.  That means that the application is always using the 
latest version of minu without re-dropping.  This also means that you 
can import minu and its module into interactive Python or IDLE's shell, 
without touching sys.path.

-- 
Terry Jan Reedy