Project organization and import

[Bruno Desthuilliers]

Martin Unsal a écrit :
> On Mar 5, 12:45 am, "bruno.desthuilli... at gmail.com"
> <bruno.desthuilli... at gmail.com> wrote:
> 
>>Remember that you can put code in
>>the __init__.py of a package, and that this code can import sub-
>>packages/modules namespaces, making the package internal organisation
>>transparent to user code
> 
> 
> Sure, but that doesn't solve the problem.
> 
> Say you have a package "widgets" with classes ScrollBar, Form, etc.
> You want the end user to "import widgets" and then invoke
> "widgets.ScrollBar()". As far as I know there are only two ways to do
> this, both seriously flawed: 1) Put all your code in one module
> widgets.py, 2) use "from scrollbar import *" in widgets/__init__.py,
> which is semi-deprecated 

"deprecated" ? Didn't see any mention of this so far. But it's bad form, 
since it makes hard to know where some symbol comes from.

# widgets.__init
from scrollbar import Scrollbar, SomeOtherStuff, some_function, SOME_CONST

> and breaks reload().

> 
>>Also remember that you can "import as", ie:
>>
>>import some_package.some_subpackage.some_module as some_module
> 
> 
> Sure but that doesn't eliminate the unfortunate interaction between
> Python class organization and filesystem heirarchy.

*class* organization ? It's not Java here. Nothing forces you to use 
classes.

> For example, say
> you want to organize the widgets package as follows:
> 
> widgets/scrollbar/*.py
> widgets/form/*.py
> widgets/common/util.py
> 
> Other than messing around with PYTHONPATH, which is horrible, I don't
> see how to import util.py from the widget code.

Some of us still manage to do so without messing with PYTHONPATH.

> 
>>Bad form IMHO. Packages and module names should be all_lower,
>>classnames CamelCased.
> 
> 
> You're still stuck doing foo.Foo() everywhere in your client code,

from foo import Foo

But:
> which is ugly 

It's not ugly, it's informative. At least you know where Foo comes from.

 > and wastes space,

My. Three letters and a dot...

> or using "from foo import *" which is
> broken.

cf above.

> 
>>>but I
>>>believe this is categorically the wrong thing to do in large projects.
>>
>>Oh yes ? Why ?
> 
> 
> For myriad reasons, just one of them being the one I stated -- smaller
> files with one functional unit each

Oh. So you're proposing that each and any single function goes in a 
separate file ?

> are more amenable to source code
> management with multiple developers.

This is not my experience.

> We could discuss this till we're blue in the face but it's beside the
> point. For any given project, architecture, and workflow, the
> developers are going to have a preference for how to organize the code
> structurally into files, directories, packages, etc. The language
> itself should not place constraints on them. The mere fact that it is
> supposedly "Pythonic" to put more functionality in one file indicates
> to me that the Python package system is obstructing some of its users
> who have perfectly good reasons to organize their code differently.

It has never been an issue for me so far.

> 
>>>you're going to want files to contain the smallest
>>>practical functional blocks. I feel pretty confident saying that "put
>>>more stuff in one file" is the wrong answer, even if it is the
>>>Pythonic answer.
>>
>>Is this actually based on working experience ? It seems that there are
>>enough not-trivial Python projects around to prove that it works just
>>fine.
> 
> 
> Yes. I've worked extensively on several projects in several languages
> with multi-million lines of code

I meant, based on working experience *with Python* ? I've still not seen 
a "multi-million" KLOC project in Python - unless of course you include 
all the stdlib and the interpreter itself, and even then I doubt we get 
so far.

> and they invariably have coding
> styles that recommend one functional unit (such as a class), or at
> most a few closely related functional units per file.

Which is what I see in most Python packages I've seen so far. But we may 
not have the same definition for "a few" and "closely related" ?

> In Python, most of the large projects I've looked at use "from foo
> import *" liberally.

I've seen few projects using this. And I wouldn't like having to 
maintain such a project.

> I guess my question boils down to this. Is "from foo import *" really
> deprecated or not?

This syntax is only supposed to be a handy shortcut for quick testing 
and exploration in an interactive session. Using it in production code 
is considered bad form.

> If everyone has to use "from foo import *"

I never did in 7 years.

> despite
> the problems it causes, how do they work around those problems (such
> as reloading)?

Do you often have a need for "reloading" in production code ???

Martin, I'm not saying Python is perfect, but it really feels like 
you're worrying about things that are not problems.