Tool for Newbz (& beyond?): ModuleWriter

[Isidor]

Hello Pythonians -

Here is a little tool I promised in a previous message....

Context/Justification:

I have just started working with Python. As I worked my way
through the tutorial, I fired up IDLE and started experimenting
with modules and some ideas of my own. Fairly quickly I had
hundreds, if not thousands of lines of attribute/method/data
testing, nascent function definitions, working functions, etc.
Invariably, I would do something that would send Python off into
never-never-land. Crash, boom. Ok, so I had saved the session as
"my long and involved session trying to use rfc822.txt." But of
course, when I opened it up in IDLE again, it was just dead text.
All those interwoven functions and stored values and imported
modules...all gone. Somehow I needed to rescue them from the
background noise of fiddling with slices and other such stuff. I
could probably do this rescue work with some extensive and
repetitive searching/copying/pasting, but I figured, hey, I'm
going to be doing this a lot, and hey, this is, among other
things, a scripting language, so....why not write a script to do
it.

So, if you have big, nasty, messy text files of IDLE sessions,
and you would like to rescue the juicy bits, this may be the tool
for you:

ModuleWriter.py

(This may be a bad name for this beast. I am considering
"ModuleMaker," "Modulizer," "module_o_matic," ;) "modOmatic,"
etc. Your suggestions will be happily considered.)


What it does:

ModuleWriter asks you (at a command prompt, no windows yet) for
an input file and an output file (and a sorting method). It then
scans your input file (a saved IDLE session...even one that is
currently open) and saves into various lists and dictionaries the
following things:

(a)    "import lines": lines that start out with "import" or
  "from [blah import blah]"
(b)    "CodeChunks": chunks of code that start with "def blah("
  (function definitions) or "class blah:".

Then it creates the output file which you specified. At the top
of the output file it puts some info, then the import lines, and
then the code chunks. You can choose which of the two existing
sorting methods to use for the code chunks:

(a)    'earliest': Sort the chunks by the earliest line number
  upon which a version of a given chunk appears.
(b)    'latest': Sort the chunks by the latest line
  number...etc...

For example:

If the code chunks in the source file are arranged like this:

def a():   #version 1
     ...
def a():   #ver 2
     ...
def b():
     ...
def c():
     ...
def a():   #ver 3
     ...

When sorted by 'earliest,' the output order will be: a, b, c.
Sorted by 'latest,' it will be: b, c, a.


NOTE 1: At present, the output includes only the *latest* version
of a function or class (as determined by the line number). In
other words, a func or class has a unique name, any code chunk
that has that same name will be considered a version of that func
or class. The version is identified by the line-number upon which
it appears. For any given func or class, ModuleWriter picks the
version with the highest line number. THIS IS A PROBLEM IF: you
had a nice working version of a function that you then fiddled
with and broke...such that later versions work more poorly than
earlier versions. Referring to the example above, only "version
3" of function "a" will appear in the output file. (See "What It
Will Be Able to Do With More Work" below for solutions to this
problem.)


NOTE 2: This tool does not attempt to produce an executable
module as its output. In fact, if you do try to execute the
output, you will find that nothing happens. This is not a missing
feature, this is a safety precaution. I don't know about you, but
I have a lot of pointless test functions floating around in my
IDLE sessions. I don't want them accidentally executed. So, after
running ModuleWriter, open up the output file, delete the
unwanted functions and classes, the repeated import statements,
etc., and then get rid of all the ">>> "s. The search and replace
dialog makes this last task incredibly easy.


GETTING IT:

I don't have web space where I can put this right now, so e-mail
me at rodisi01 at my-deja.com, and every day or every other day, I
will send it off to whoever has requested it. I know this is a
hassle, I'm sorry. If it looks like there is serious interest in
this thing, I will find some web space for it so you can click
and save to your heart's content. In fact, I thought I heard
someone in this list talk about offering web space for python
projects....;)...

ALTERNATIVELY:

If someone will point me to (or give me) some suggestions re.
posting code to the newsgroup, I could just do that. I'm worried
about line breaks and all that stuff. Would an ascii attachment
work in deja-world? (I've never had much access to newsgroups, so
this is all sort of new to me, and I have to use the free and
powerful (many many thanks!) but somewhat frustrating deja-news
interface.)


DISCLAIMER (and request for comments):

This is the first module I have written. It is clumsy and ugly.
It is not optimized. There's a lot of python I know nothing about
yet, etc. etc. However, it works. Since it's not the kind of
thing that you run a thousand times a day, its poor performance
is not a major problem for the time being. I find it very useful
even in its present, crude incarnation, and I suspect other
newbies may do so as well. So, I am releasing it as it is for
now, with plans to make it better as I go. Of course, comments
and suggestions will make the improvement process go much more
quickly! ;) ...Even something as simple as: hey, you should think
about using module x instead of module y for line q/ function z.


What It Will Be Able To Do With More Work (i.e, To Do):

1.   Easy:

     a)   Make it take arguments from the command line: e.g.,
          ModuleWriter(inputfile, outputfile, sortversion) .
     
     b)   Add a graphical interface. I am working on figuring out
          Tkinter now. As soon as I think I can pull it off, I'll
          add a Tkinter interface.

2.   Not as easy, but useful:

     Add support for documentation. I'm working on this already.
     What I mean by this is that ModuleWriter will find and
     manage lines from your IDLE session that start with things
     like: "##NoteLocal: ", "##NoteGlobal: ", "##FAQ: ",
     "##Example: ", etc. I need to figure out what it should do
     with such lines, but I think this will be really nice. Some
     should go in the requested output file...as notes about
     functions and classes. Some should go in a globally selected
     "faq" file or whatever, so that every session you do, you
     can add to a personal faq that eventually can be shared with
     others....etc etc etc.

3.   Harder but even more interesting:

     All the versions of code chunks get stored during file
     processing. With a nice little gui "dictionary browser"
     (something like the IDLE class browser?), you could view any
     of the different versions and select which ones you want in
     the output. Such a "dictionary browser" could also be used
     to browse the faq file created by ModuleWriter. etc.


Again, any suggestions, advice, etc. towards these ends will be
gratefully accepted.


Ok, I hope this is useful to someone out there, it sure is for
me!

Take care...

Isidor