Managing import statements

[Chris Mellon]

On 12/10/05, Shane Hathaway <shane at hathawaymix.org> wrote:
> Let's talk about the problem I really want help with.  I brought up a
> proposal earlier, but it was only half serious.  I realize Python is too
> sacred to accept such a heretical change. ;-)
>
> Here's the real problem: maintaining import statements when moving
> sizable blocks of code between modules is hairy and error prone.
>
> I move major code sections almost every day.  I'm constantly
> restructuring the code to make it clearer and simpler, to minimize
> duplication, and to meet new requirements.  To give you an idea of the
> size I'm talking about, just today I moved around 600 lines between
> about 8 modules, resulting in a 1400 line diff.  It wasn't just
> cut-n-paste, either: nearly every line I moved needed adjustment to work
> in its new context.
>

You might want to invest in refactoring tools, like the ever-popular
Bicycle Repair Man, which can help ensure that you rename things
correctly.

> While moving and adjusting the code, I also adjusted the import
> statements.  When I finished, I ran the test suite, and sure enough, I
> had missed some imports.  While the test suite helps a lot, it's
> prohibitively difficult to cover all code in the test suite, and I had
> lingering doubts about the correctness of all those import statements.
> So I examined them some more and found a couple more mistakes.
> Altogether I estimate I spent 20% of my time just examining and fixing
> import statements, and who knows what other imports I missed.
>

Both PyLint and PyChecker can detect missing imports (as well as
redundant ones) for you.

> I'm surprised this problem isn't more familiar to the group.  Perhaps
> some thought I was asking a newbie question.  I'm definitely a newbie in
> the sum of human knowledge, but at least I've learned some tiny fraction
> of it that includes Python, DRY, test-first methodology, OOP, design
> patterns, XP, and other things that are commonly understood by this
> group.  Let's move beyond that.  I'm looking for ways to gain just a
> little more productivity, and improving the process of managing imports
> could be low-hanging fruit.
>

It is probably because most people don't regularly switch that much
code around, or use that many modules. I think the fact that you move
that much code between modules is probably a code smell in and of
itself - you're clearly moving and changing responsibilities.
Refactoring in smaller chunks, less extreme refactoring, correcting
the imports as you go, and avoiding inline imports without a really
good reason will probably help you a lot more than a new syntax.

> So, how about PyDev?  Does it generate import statements for you?  I've
> never succeeded in configuring PyDev to perform autocompletion, but if
> someone can say it's worth the effort, I'd be willing to spend time
> debugging my PyDev configuration.
>

PyDev will not generate import statements for you.

> How about PyLint / PyChecker?  Can I configure one of them to tell me
> only about missing / extra imports?  Last time I used one of those
> tools, it spewed excessively pedantic warnings.  Should I reconsider?
>

Yes. Spend an afternoon looking at PyLints options, get rid of the
stuff you don't like, and use it regularly. PyDev has PyLint
integration, which is nice.

> Is there a tool that simply scans each module and updates the import
> statements, subject to my approval?  Maybe someone has worked on this,
> but I haven't found the right Google incantation to discover it.
>

Assuming that you keep your global namespace pure, then you can do
this yourself pretty simply. If you don't, there is no way for
anything to know that you want symbol foo from module bar and not from
module baz.

I don't think I've ever imported more than a dozen modules into any
given file. I rarely find it neccesary to move huge chunks of code
between my modules - the biggest such I did was when I moved from a
single-file proof of concept to a real modular structure, and that
only took me an hour or so. If I'd done it modularly to start with it
would have been fine.

> Shane
> --
> http://mail.python.org/mailman/listinfo/python-list
>