[Tutor] Debugging skills

[dn]

On 16/01/24 05:16, Mats Wichmann wrote:
...

>> If you come across someone who lands in just such a problem, where 
>> his/her mental model is considerably at-odds with the interpreter; it 
>> would be very interesting to see if/how an LLM helps them through.
>>
>> Alternately, if you can recall such circumstance, I'll be keen to try 
>> to reproduce and to experiment...
> 
> Well, I can name one very simple one:  if you've programmed only in, for 
> example C, your mental model is arrays are pre-allocated (and hopefully 
> you remember the fixed size so you don't go out of bounds), so you might 
> try to use a Python list the same way - the syntax looks similar, after 
> all...
> 
> data = []
> for i in range(5):
>      data[i] = i
> 
> IndexError: list assignment index out of range
> 
> Wait, what?
> 
> Obviously you then quickly learn, so I'm not sure this was a 
> particularly useful example, but the brain wants to follow patterns it 
> knows.

Not at all. the phrase "brain...patterns" is the cognitive issue - 
although this disconnect has been framed in the context of someone who 
already knows how to program[me] (in another language) and is adding 
Python to those skills.
(slightly different situation to someone programming for the first time!)

However, don't most languages which allow such code, also require the 
array to be pre-defined, eg def data[ 5 ]?

If so, the coder has (apparently) casually disregarded the implications 
of removing that!


Regardless, have 'interviewed' Anthropic's Claude2 and OpenAI's ChatGPT 
3.5, and will send their responses in separate messages to facilitate 
side-by-side viewing.

The experiment is compromised! I asked the question on behalf of 
A.N.Other, for fear of the system habitually down-grading the 
level/quality of answer it will give to me in other experiments. Thus, 
felt the need to add an extra request for explanation, which improved 
the response in both cases - but required more work from 'me'. Would a 
trainee's personal phrasing elicit a better response, earlier?


What do you think of the way the LLMs have responded to this scenario?
(am asking you, gentle reader, not only Mats as 'challenger'!)


-- 
Regards,
=dn