"False exceptions?" (was Re: theme of the week: tools

[David Bolen]

"Dan Perl" <danperl at rogers.com> writes:

> I'm new to wxPython so I cannot relate yet to what you are saying.  The 
> wxPython extension module may catch the exceptions and suppress them, but 
> doesn't it generate a message for that or something?  I hope that they are 
> not hiding the exception completely (and that they generate something) and 
> then you can still debug the problem with well placed breakpoints.  Am I 
> wrong here or is there something I'm missing?

Yes, the wxPython layer will generally output an exception, but only
if there's a console available to receive it.

Even if you do have a console showing exceptions, not all exceptions
are cleanly reproduceable or predictable.  I may have code that
typically runs just great, but that occasionally generates an
exception.  Breakpoints may not work if the code runs many times
successfully before failing (due to varieties in calling parameters at
runtime).  Perhaps a watchpoint might work but you'd have to know
enough about the failure to establish the watchpoint parameters.

Instead, with Wing, just start it up, let it run (or if you need to,
run it independently of Wing but with the stub loaded) until the
exception occurs.  No hassles or up front work.

> The false positive that I encountered was indeed in 'sre', invoked by 
> pickle.  I think it was an IndexError which should be a normal thing if they 
> are using it to stop at the end of an iteration.  So I'm not convinced this 
> would constitute "minimal" false positives.

Did you get any others?  I was talking "minimal" in terms of volume,
not necessarily significance (which may be more subjective).  I didn't
have much of an issue choosing to ignore the sre case (after which it
never showed up again for my project).

> Maybe if I would have been convinced first with a personal experience about 
> the advantages of Wing's exception detection at the point of raise, then I 
> would have accepted the false positives more easily.  But seeing the false 
> exception first (actually right away and with positive spins on it) and then 
> finding out the right technical explanation later, it made it much harder to 
> swallow.

That could well be.  When I was testing Wing originally being able to
effectively work with legacy wxPython apps was a high priority for me,
both in terms of the auto-completion and in terms of exception
handling (the latter of which I had been disappointed to find not
working in other IDEs I had tried to that point).  So I was probably
predisposed to not worry too much about the side-effects of
functionality that at least got me the exception processing I wanted.

(...)
> I would need to first see an example of generating an exception without 
> knowing it and that exception still being relevant although it is being 
> handled.

I'd split that into two parts - (a) generating the exception without
knowing it, and (b) it being relevant even if handled.

For (a), I can't seem to come up with a good concise example, so maybe
I'll just agree that you may know about, but not be able to act on it
in a useful fashion.  (The cases I was thinking of were GUI apps
without a console, or apps generating output where a traceback might
be missed, but I can come up with reasonable solutions in those cases
if you knew you were looking for an exception, so I'm willing to bet
you can too.  The other case was with a locally developed extension, but
that's susceptible to the argument of just fixing the local extension :-))

Note however, that knowing about it in many cases may only be through
a stderr traceback, so to know it other than manually inspecting it at
runtime would require trapping stderr in general and logging that
somewhere, and then having something to recognize tracebacks in that
log.

(b) is certainly possible though, although at the risk of overuse I'll
fall back to wxPython again.  In wxPython any exception that occurs in
an event handler will be handled by the wxPython extension, and is
thus invisible to top level code.  So if you have an exception in an
event handler, it interrupts the exception handler, which presumably
no longer does what it is supposed to, so that's relevant.  The
exception will not, however, stop the application, nor bubble up to
any top level application exception handler.  The only place you'd see
an indication of it would be in the traceback to stderr - your
application has no opportunity to handle it otherwise (unless you have
exception handlers in each individual event handler).

Now in normal operation that's actually a very attractive part of
wxPython.  By and large you can create problems, and the application
keeps running.  Perhaps one element of the GUI might not react
properly or some button won't work, but the application survives.  But
when trying to troubleshoot that problem, it's often most convenient
to interact with the application as a user until the problem occurs,
letting the debugger simply catch things at the point when it happens.

>           I can imagine something not working right and an exception being 
> one more indication for the problem, but then there are still other good 
> ways to debug the problem.  

I'd certainly agree with that.  As with everything, the debugger is
just one element in the toolkit.  But even if you have an exception
being generated, there is real value to being able to stop and
dynamically inspect the state of the system at the point of exception,
particularly when an exception is sporadic or difficult to reproduce.

Having other methods to also debug it doesn't negate the value of having
a debugger that can run an application until such an event takes place and
then give you direct access to the state of the system at that point.

>                             And if things are working, I don't see how 
> hidden exceptions may be of interest.  I have very little experience with 
> GUIs but I realize that unit testing is much more difficult for GUIs.  Does 
> that make it necessary to "hunt for exceptions" like that instead of testing 
> for results correctness?  It may also be just my short experience with 
> Python and especially my total lack of experience with complex systems in 
> Python (e.g, Zope), so I'm ready to listen to an argument on that point.

I definitely have multiple categories of code.  New projects that can
make use of TDD approaches do so and have far less of a requirement
(IMO so far) of debuggers in general.  They're generally also more
amenable to "print" debugging since issues arise that tend to be more
focused, and occurring in well identified areas of the code under well
identified conditions.

Older projects not written in a TDD fashion, or those for which unit
testing is difficult (in my environment typically GUI, threading, and
network in some mixture) often run into scenarios where you need a
lengthy runtime or real world usage to tickle sporadic or
non-deterministic bugs.  In those cases, a solid debugger with the
ability to stop at any moment when a problem arises can be worth its
weight in gold - even if such occurrances may be few and far between.
I believe Alex Martelli recently posted about such a case in a web
application, for example, where a debugger ended up very useful in
addition to the other techniques being applied.

-- David