Beginners and experts (Batchelder blog post)

[Steve D'Aprano]

On Fri, 29 Sep 2017 03:28 pm, Gregory Ewing wrote:

> Chris Angelico wrote:
>> finding the bug is basically searching
>> through a problem space of all things that could potentially cause
>> this symptom. A novice could accidentally stumble onto the right
>> solution to a tricky bug, or an expert could search a thousand other
>> things and only get to the true cause after a long time.
> 
> What I think is more important than how *long* it takes is
> *how* they go about finding the bug.
> 
> A novice will make wild guesses about what might be wrong
> and make random changes to the program in the hope of
> making it work. An experienced programmer will conduct
> a series of carefully-designed experiments to narrow
> down the cause.

"Carefully-designed experiments" -- yeah, that is so totally how the coders I've
worked with operate *wink*

I think that's an awfully optimistic description of how the average programmer
works :-)


> A result of this is that a true expert will never have
> to try a thousand possibilities. He will be able to
> search a space of N possible causes in O(log N) operations.

I don't think that, *in general*, possible causes of a bug can be neatly sorted
in such a way that we can do a binary search over the space of all possible
causes.

I also think it is unlikely that even the best programmer enumerates all the
possible causes before working on them. If you don't enumerate all the causes
first, how to you sort them?

In my experience, even good coders are likely to say "there's only three
possible things that could be causing this bug", and later on report it was
actually the fifth thing.

More likely, the experienced programmer is better at eliminating irrelevancies
and narrowing in on the space of likely candidates, or at least narrowing down
on the region of code that is relevant, while less experienced programmers
waste more time looking at things which couldn't possibly be the cause[1].

More likely, the experienced programmer makes better use of his or her tools.
While the novice is still messing about trying to guess the problem by pure
logical reasoning, the expert is using a few useful print calls, or a debugger,
to narrow down to where the problem actually is.

And experts are likely to be better at extrapolating "well, since that's not the
problem... maybe its this?". And you learn that not by pure logic, but by being
bitten by nasty bugs:

"I can't see any possibly way that this could be involved, but I came across a
similar situation once before and this was the solution... [confirms hypothesis
and fixes bug] oh of course, that's how the bug occurs! Its obvious in
hindsight."

More experienced programmers are confident enough to know when to walk away from
a problem and let their subconscious work on it. Or when to go and explain it
to the cleaning lady[2]. Or when to admit defeat and hand over to a fresh pair
of eyes who won't be stuck in the same mindset.

Even just getting to the point of being able to reason *where to start* requires
a fair amount of experience. A true beginner to programming altogether won't
even know how to read a stack trace to identify the line where the bug occurs,
let alone understand why it happened. I wish I had a dollar for every time some
beginner says something like:

"I get a syntax error. What's wrong with my code?"

and eventually after much arm-twisting is convinced to post the last line of the
traceback, which turns out not to be a syntax error at all:

"TypeError: object of type 'int' has no len()"

(say). Reading error messages is a skill that must be learned, even in Python.
Let alone (say) gcc error messages, which are baroque to an extreme. The other
day I was getting an error like:

/tmp/ccchKJVU.o: In function `__static_initialization_and_destruction_0(int,
int)':
foo.cpp:(.text+0x7c): undefined reference to `std::ios_base::Init::Init()'
foo.cpp:(.text+0x91): undefined reference to `std::ios_base::Init::~Init()'
collect2: error: ld returned 1 exit status


Of course, its obvious that the problem here is that I needed to install g++ as
well as gcc, right? :-)


> This doesn't necessarily translate into time, because
> in some situations the experiments can be very time-
> consuming to perform. But other things being equal,
> the expert's bug-finding algorithm is faster than the
> novice's.





[1] Although, the *really* experienced programmer knows that in sufficiently
baroque and highly-coupled code, a bug could be caused by *anything*
*anywhere*. (This is one reason why global variables are bad.)

I still haven't gotten over hearing about a bug in the Internet Explorer
routines for handling WMF files, which lead to being unable to copy and paste
plain text in any application.


[2] The place I worked had a cuddly penguin toy called Mr Snuggles, and the
programmers would go and explain the problem to him. It never[3] failed.

[3] Well, hardly ever.


-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.