Organising packages/modules - importing functions from a common.py in a separate directory?

[Victor Hooi]

Hi,

I have a collection of Python scripts I'm using to load various bits of data into a database.

I'd like to move some of the common functions (e.g. to setup loggers, reading in configuration etc.) into a common file, and import them from there.

I've created empty __init__.py files, and my current directory structure looks something like this:

foo_loading/
    __init__.py
    common/
        common_foo.py
    em_load/
        __init__.py
        config.yaml
        sync_em.py
    pg_load/
        __init__.py
        config.yaml
        sync_pg.py

So from within the sync_em.py script, I'm trying to import a function from foo_loading/common/common_foo.py.

    from ..common.common_foo import setup_foo_logging

I get the error:

    ValueError: Attempted relative import in non-package 

If I change directories to the parent of "foo_loading", then run

    python -m foo_loading.em_load.sync_em sync_em.py

it works. However, this seems a bit roundabout, and I suspect I'm not doing things correctly.

Ideally, I want a user to be able to just run sync_em.py from it's own directory, and have it correctly import the logging/config modules from common_foo.py, and just work.

What is the correct way to achieve this?

Secondly, if I want to move all of the config.yaml files to a common foo_loading/config.yaml, or even foo_loading/config/config.yaml, what is the correct way to access this from within the scripts? Should I just be using "../", or is there a better way?

Cheers,
Victor