File handling: The easy and the hard way

[Hans-Joachim Widmaier]

Am Thu, 30 Sep 2004 10:37:49 -0400 schrieb Steve Holden:


> I agree we really shouldn't expect users to have to see tracebacks, but 
> that doesn't mean that exception handling has to be sophisticated.
> 
> Here's something I'd consider acceptable, which doesn't add hugely to 
> the programming overhead for multiple files but doesn't burden the user 
> with horrible tracebacks. I've used it to print itself, so you see how 
> it works and what it contains all in the same output:
> 
> sholden at DELLBOY ~
> $ ./ft.py one.py ft.py
> Problem handling file one.py : [Errno 2] No such file or directory: 'one.py'

Yes, this should be acceptable for the simple scripts I mentioned.

> #!/usr/bin/python
> #
> # ft.py: simple multi-file processor with error handling # import sys
> 
> files = sys.argv[1:]
> 
> for f in files:
>      try:
>          for l in file(f):
>              sys.stdout.write(l)
>      except Exception, reason:
>          print >> sys.stderr, "Problem handling file", f, ":", reason

Looks very familiar, but doesn't handle '-' (stdin) (but that's another
story, albeit somewhat related). ;-)

> I'm quite happy to let the process-termination housekeeping code, or
> perhaps (in some implementations) the Python housekeeping at garbage
> collection, close the file, which you might think is unduly sloppy. What
> can I say, the user pays if they don't want sloppy :-). But I'd consider
> this sufficiently close to "production quality" to be delivered to
> end-users. Clearly you can add file assignment to a variable and a
> try/finally to ensure it's closed. You gets what you pays for.

I often do that, too. But every time I do it, I feel guilty. (I did my
first somewhat more serious programming on an Amiga, where the OS did
_not_ clean up after you - which formed a habit of religiously keeping
track of and freeing each and every resource in each and every case.)

> Well, the more complex your processing gets the more complex your
> error-handling gets too,

This doesn't come as a surprise. :-) Yet I can see a class of basically
simple programs that are nonetheless meant to be of "production quality"
and where _most_ of the complexity stems from handling these errors.

> but I'd say you should look at some serious refactoring here - you
> appear to have what's sometimes called a "code smell" in extreme
> programming circles. See
>
>      http://c2.com/cgi/wiki/?CodeSmell

Never heard of that before. ;-) Anyway, maybe my understanding of
'refactoring' is wrong, but isn't my desire to 'factor out' the error
handling so I can make it rigid/good and reusable (a good incentive for
me!) what might be called refactoring? Cluttering every script with always
the same error handling pattern, well, does have a smell to me. These
things are just too common to not being solved once and hidden in, e.g.,
a module.


> The problem that any such approach is likely to have can be summed up as
> "If processing is complicated then error-handling may also become
> complicated, and error-recovery even more so". You shouldn't expect it
> to be too simple, but if it's too complex then you might find that a
> restructuring of your program will yield better results.

I do not think it'd be too simple. I think I can (mostly) cope with the
complexity. I'm disappointed because I haven't yet found a nice solution
where I can _hide_ this complexity and do not have to slap it in every
file that starts with '#!/usr/bin/env python'. Still dreaming of a File
object that does all this, maybe with a helper module "fileerrhandler".
Granted, the name is ugly. ;-)

["The Mythical Man-Month"]

> The combination of all three components is to be found in beasts
> sometimes known as "software engineers", frequently held by some to be
> mythical.

Now this is a title that can never be applied to me. I'm basically a
hardware designer, mostly writing embedded programs and doing system
administration by now. And I must admit that I never really *learned*
programming. I'm not in the same league as most others here, so I'm always
uncertain whether my gut feeling (nose tingling?) is correct or totally
off track.

> If I were feeling cynical, I might sum this up by saying "Python is a
> programming language, not a f***ing magic wand". But that won't stop
> people from looking for the silver bullet that solves all their problems
> in a songle line of code.

I daresay that if 'sprintf()', 'gets()' and the like never existed, there
would be a lot less CERT advisories. Or, if a file type enforces sane
error handling, there would be less tracebacks. :-)

> Hope this helps, and doesn't come across as critical. Your questions are
> reasonable, and show a sincere appreciation of the difficulties of
> producing high-quality software. And we don't ever want anything else,
> do we?

I'm very grateful for your detailed reply. clp always strikes me as being
a place where you don't get smart-ass replies but elaborate answers that
sometimes look like articles in a classy magazine.

Thank you very much for your time and suggestions,
Hans-Joachim