Organising packages/modules - importing functions from a common.py in a separate directory?

[Victor Hooi]

Hi,

Wait - err, subpackage != module, right? Do you think you could explain what a sub-package is please? I tried Googling, and couldn't seem to find the term in this context.

Also, so you're saying to put the actual script that I want to invoke *outside* the Python package.

Do you mean something like this:

> sync_em.py
> sync_pg.py
> foo_loading/ 
>     __init__.py 
>     common/ 
>         common_foo.py 
>     em_load/ 
>         __init__.py 
>         config.yaml 
>         em.py
>     pg_load/ 
>         __init__.py 
>         config.yaml 
>         pg.py

and the sync_em.py and sync_pg.py would just be thin wrappers pulling in things from em.py and pg.py? Is that a recommended approach to organise the code?

Would it make any difference if I actually packaged it up so you could install it in site-packages? Could I then call modules from other modules within the package?

Cheers,
Victor

On Tuesday, 29 October 2013 18:44:47 UTC+11, Peter Otten  wrote:
> Victor Hooi wrote:
> 
> 
> 
> > Hi,
> 
> > 
> 
> > Hmm, this post on SO seems to suggest that importing from another sibling
> 
> > directory in a package ins't actually possibly in Python without some ugly
> 
> > hacks?
> 
> > 
> 
> > http://stackoverflow.com/questions/6323860/sibling-package-imports
> 
> > 
> 
> > Did I read the above correctly?
> 
> 
> 
> Yes.
> 
>  
> 
> > Is there another way I can structure my code so that I can run the
> 
> > sync_em.py and sync_pg.py scripts, and they can pull common functions from
> 
> > somewhere?
> 
> 
> 
> The packages you are trying to access in your original post 
> 
> 
> 
> > foo_loading/
> 
> >     __init__.py
> 
> >     common/
> 
> >         common_foo.py
> 
> >     em_load/
> 
> >         __init__.py
> 
> >         config.yaml
> 
> >         sync_em.py
> 
> >     pg_load/
> 
> >         __init__.py
> 
> >         config.yaml
> 
> >         sync_pg.py
> 
> 
> 
> 
> 
> aren't actually siblings in the sense of the stackoverflow topic above, they 
> 
> are subpackages of foo_loading, and as you already found out
> 
> 
> 
> > So from within the sync_em.py script, I'm trying to import a function from 
> 
> foo_loading/common/common_foo.py.
> 
> > 
> 
> >     from ..common.common_foo import setup_foo_logging
> 
> > 
> 
> > I get the error:
> 
> > 
> 
> >     ValueError: Attempted relative import in non-package 
> 
> > 
> 
> > If I change directories to the parent of "foo_loading", then run
> 
> > 
> 
> >     python -m foo_loading.em_load.sync_em sync_em.py
> 
> > 
> 
> > it works. However, this seems a bit roundabout, and I suspect I'm not 
> 
> doing things correctly.
> 
> > 
> 
> > Ideally, I want a user to be able to just run sync_em.py from it's own 
> 
> directory, and have it correctly import the logging/config modules from 
> 
> common_foo.py, and just work.
> 
> > 
> 
> > What is the correct way to achieve this?
> 
> 
> 
> you can access them as long as the *parent* directory of foo_loading is in 
> 
> sys.path through PYTHONPATH, or as the working directory, or any other 
> 
> means. However, if you step into the package, e. g.
> 
> 
> 
> $ cd foo_loading
> 
> $ python -c 'import common'
> 
> 
> 
> then from Python's point of view 'common' is a toplevel package rather than 
> 
> the intended 'foo_loading.common', and intra-package imports will break.
> 
> 
> 
> To preserve your sanity I therefore recommend that you 
> 
> 
> 
> (1) avoid to put package directories into sys.path
> 
> (1a) avoid to cd into a package
> 
> (2) put scripts you plan to invoke directly rather than import outside the 
> 
> package.