Project organization and import

[Chris Mellon]

On 5 Mar 2007 23:35:00 -0800, Martin Unsal <martinunsal at gmail.com> wrote:
> On Mar 5, 11:06 am, "Chris Mellon" <arka... at gmail.com> wrote:
> > I never advocated big files with many functional units - just files
> > that are "just big enough".
>
> Then we're in total agreement. I'm not sure why you thought my
> opinions were the result of baggage from other languages when you
> don't seem to actually disagree with me.
>

Because you're advocating single class per file. A scan through the
standard library may be instructive, where there are some modules that
expose a single class (StringIO, pprint) and others that expose many,
and some that expose none at all. "smallest unit that it makes sense
to work on" and "single class" are totally different things. In any
case, as I hinted at, I prefer an organic, developer driven approach
to deciding these things, not handed down from above style guidelines.
You know your modules are broken up enough when you no longer have
conflicts.

> > Fewer dependencies between compilation units means a
> > faster rebuild-test turnaround.
>
> I know all about incremental builds and I just don't think people use
> small compilation units in C++ to make their builds faster. It
> certainly never been the reason why I subdivided a source file.
>

Faster compile/debug/edit cycle is the main justification I've heard
for single class per file. The others are variations of your RCS
argument, which I don't think is justifiable for the above reasons. It
smells of the kind of "my developers are stupid" short sighted
management that kills projects.

> > Sure, but whats your goal here? If you're just testing something as
> > you work, then this works fine. If you're testing large changes, that
> > affect many modules, then you *need* to reload your world, because you
> > want to make sure that what you're testing is clean.
>
> I don't think reload works for anything but trivial scripts. The
> moment you use "from foo import bar" reload is broken.
>
> > The semantics of exactly what reload should do are tricky. Pythons
> > reload works in a sensible but limited way.
>
> I agree that there is some subtlety there, and I appreciate your
> example. However the fact that Python's module system essentially
> forces you to use "from foo import *" and that reload is almost
> entirely imcompatible with "from foo import *", I would say that
> reload is essentially useless.
>

I'm still not sure why you believe this, since several counterexamples
where given. As an intellectual exercise, though, lets assume that
reload is totally broken and you just can't use it. Pretend it will
reformat your machine if you ever call it. Can you really think of no
other reason to use Python? You still haven't given any justification
for why a magic reload is essential to Python development when a) all
existing python development works fine without it and b) all existing
development in every other language works fine without it.

> > That said, nothing prevents you from using "from foo import Foo" if
> > Foo is all you need (or need most - you can combine this with import
> > foo).
>
> Well "from foo import Foo" is just a special case of "from foo import
> *". :) It still breaks reload. It still means you're restarting your
> interpreter even to do the most trivial development cycle.
>

You're totally fixated on reload. I don't understand this. I'm totally
positive that your traditional development experience has not been in
an environment where you could effortlessly slot in new code to a
running image. Why do you demand it from Python?

Also, the difference between "from foo import Bar" and "from foo
import *" is that the former is limited in scope (you're adding a
limited set of explicit names to your namespace) and is futureproof
(additional names exported from foo won't clash with vars in the
importing module with unknown effects). The reason why one is common
and accepted and the other is frowned upon has nothing to do with
reload().

> > I wonder what environments you worked in before that actually had a
> > reliable and gotcha free version of reload?
>
> I'm perfectly well aware that I'm not going to be able to reload a
> widget in the middle of a running GUI app, for example. I'm not
> looking for gotcha free, I'll settle for minimally useful.
>

Then reload() as is is what you want.

> Here's an analogy. In C, you can do an incremental build and run your
> modified application without having to first reboot your computer. In
> Python, where reload() is essentially the incremental build process,
> and the interpreter is essentially a virtual machine, you guys are
> saying that my best option is to just "reboot" the virtual machine to
> make sure I have a "clean slate". It may be the path of least
> resistance, but to say that it is necessary or inevitable is 1960s
> mainframe thinking.
>

But you do need to restart the application image. The python
interpreter is not an emulator. You're drawing incompatible analogies
and making unjustified assumptions based on them. reload() is not an
incremental build process, and starting a new Python instance is not
rebooting your machine. This is just not a justifiable comparison.