Managing import statements

[Jean-Paul Calderone]

On Sat, 10 Dec 2005 02:21:39 -0700, Shane Hathaway <shane at hathawaymix.org> wrote:
>Let's talk about the problem I really want help with.  I brought up a
>proposal earlier, but it was only half serious.  I realize Python is too
>sacred to accept such a heretical change. ;-)
>
>Here's the real problem: maintaining import statements when moving
>sizable blocks of code between modules is hairy and error prone.
>
>I move major code sections almost every day.  I'm constantly
>restructuring the code to make it clearer and simpler, to minimize
>duplication, and to meet new requirements.  To give you an idea of the
>size I'm talking about, just today I moved around 600 lines between
>about 8 modules, resulting in a 1400 line diff.  It wasn't just
>cut-n-paste, either: nearly every line I moved needed adjustment to work
>in its new context.
>
>While moving and adjusting the code, I also adjusted the import
>statements.  When I finished, I ran the test suite, and sure enough, I
>had missed some imports.  While the test suite helps a lot, it's
>prohibitively difficult to cover all code in the test suite, and I had

I don't know about this :)

>lingering doubts about the correctness of all those import statements.
>So I examined them some more and found a couple more mistakes.
>Altogether I estimate I spent 20% of my time just examining and fixing
>import statements, and who knows what other imports I missed.
>
>I'm surprised this problem isn't more familiar to the group.  Perhaps
>some thought I was asking a newbie question.  I'm definitely a newbie in
>the sum of human knowledge, but at least I've learned some tiny fraction
>of it that includes Python, DRY, test-first methodology, OOP, design
>patterns, XP, and other things that are commonly understood by this
>group.  Let's move beyond that.  I'm looking for ways to gain just a
>little more productivity, and improving the process of managing imports
>could be low-hanging fruit.
>
>So, how about PyDev?  Does it generate import statements for you?  I've
>never succeeded in configuring PyDev to perform autocompletion, but if
>someone can say it's worth the effort, I'd be willing to spend time
>debugging my PyDev configuration.
>
>How about PyLint / PyChecker?  Can I configure one of them to tell me
>only about missing / extra imports?  Last time I used one of those
>tools, it spewed excessively pedantic warnings.  Should I reconsider?

I use pyflakes for this: <http://divmod.org/trac/wiki/DivmodPyflakes>.  The *only* things it tells me about are modules that are imported but never used and names that are used but not defined.  It's false positive rate is something like 1 in 10,000.

>
>Is there a tool that simply scans each module and updates the import
>statements, subject to my approval?  Maybe someone has worked on this,
>but I haven't found the right Google incantation to discover it.

This is something I've long wanted to add to pyflakes (or as another feature of pyflakes/emacs integration).

Jean-Paul