Development engineering (was: python ides)

[Cameron Laird]

In article <1gkipxd.1ibrh95fcvdotN%aleaxit at yahoo.com>,
Alex Martelli <aleaxit at yahoo.com> wrote:
			.
			.
			.
>I don't need one often, but sometimes one DOES run into something
>sufficiently weird -- and when it happens it can take days to sort it
>out.  Case in point: rewriting the web layer for a client's multi-tier
>application framework, to use twisted and nevow instead of Webware, we
>were occasionally getting the weirdest 'recursion limit exceeded'
>tracebacks -- and if you've ever tried to divine what's going on from a
>Twisted traceback with a few dozen deferreds in play, some intercepting
>errors and some not, etc, etc, you know that isn't comfortable.
>
>Two of us trashed around for 2-3 days trying all the usual things.
>Finally one of the GUI specialists in another subteam of the same
>framework development team offered to help us with WingIDE.  He's the
>one who does custom layout widgets, weird super-nested-lists-and-
>trees-widgets, etc etc, for the cross-platform GUI-clients of that
>framework, on top of Qt/PyQt etc, and claims that without that debugger
>he could never solve the weird and delicate issues that continually
>arise in his work.
>
>Be that as it may, in 2-3 hours we were on top of the problem: there is
>an intrinsic, undocumented limit to how many items you can feed to a
>newvow:sequence renderer, if those items are deferreds, because they all
>get chained into a linked list whose __del__, at the end, proceeds _by
>recursion_.  So, if you have too many items in the sequence, it's
>recursion-limit-exceeded time, and the dance of bouncing exception
>handlers starts playing (in a too-deep-nesting situation, too).  Once
>the problem was identified the solution was trivial (as it generally
>is): there was really no justification for showing a thousand 'hits' of
>some search as one humungous table in a single webpage anyway... we
>simply hadn't bothered yet with implementing the code to slice the hits
>into multiple pages and let the user page back and forth in the browser.
>So we did, ensuring no single sequence of hundreds of deferreds could
>ever be rendered in one gulp, and the problem disappeared.
>
>But, that debugger really DID save our bacon on that occasion.  I'm not
>sure how it managed to get control at the right time within that thicket
>of exception handlers and let us examine hundreds of nested stack frames
>looking for the underlying cause (I know the cause sounds trivially
>obvious in retrospect, but THAT, too, is par for the course... one of
>the hardest bugs to find in my career, back in Fortran days, boiled down
>in the end to an .LT. being used where an .LE. should have been... a
>*one*-*character* fix after days of fighting to find the right place out
>of over a million lines of code in the system...!-).  But manage it did.
>Next time we spend more than 2-3 hours chasing some bug that just won't
>show up, I'm going to go and ask that WingIDE expert user for some
>little help!-)
			.
			.
			.
I have an intense interest in this narrative, and little
ability yet to articulate why.

Excellent developers, among other things, diagnose accurately
and quickly; slightly less (I claim) differentiating is their
creativity at remedies.

I have a large, large theory that I summarize as, "Cut your
losses".  The local application is that one big win for
engineering productivity in software development is avoidance
or management of those it-took-*months*-to-track-down-this-
simple-problem situations.  I've sure lived through them; in
fact, I'm dealing with three different ones that I know right
now.

They are humbling, indeed.

I've lived through quite a few, as I just wrote.  I do *not*
particularly associate their resolution, in my memories of
first-hand experience, with debuggers.  Among other things,
this makes me wonder if I'm missing out on the capabilities
of debuggers.

I don't know what the answers are.  In this area, I'm still
wrestling with what questions to ask.  This stuff *fascinates*
me, though, in utter contrast to what I consider the utter
irrelevancies of which editor to use, which ...