[Idle-dev] Enhancements to IDLE debugger

[Terry Reedy]

On 10/30/2016 6:00 PM, Irv Kalb wrote:

> I am new to this list, so here’s a quick introduction.  I have been a
> software developer for over 30 years.  I’ve worked for a number of
> different companies and then worked for many years as a consultant.
> Over the course of my career, I’ve used many different languages,
> many operating systems, and many different development environments.
> I’ve worked on the development of operating systems, compilers, user
> interface work, kids edutainment software, eLearning systems and
> more.
>
> For the past six years, I’ve been teaching software development at
> three different colleges in Silicon Valley, California.  I mostly
> teach intro to programming, using Python.  Most of my students are
> not Computer Science majors, in fact, most are art students, and most
> have never done any programming before.  In my classes, I teach my
> students to use IDLE because it is installed as part of the standard
> installation on all of the school’s computers.  For these students it
> works perfectly.  It is simple for them to understand, and  there is
> no need to access the command line.

I am the current IDLE maintainer, still learning some of the darker 
corners of idlelib.  The debugger is one of them.  One of the things I 
need is more feedback from users and especially active instructors. 
Thank you for writing.

> I am writing to suggest an improvement to the IDLE environment that
> would be of great benefit to students (and to teachers, correcting
> student’s assignments).
>
> In my work as a developer, I got used to working with good debuggers.
> I use a debugger as an invaluable tool to track down bugs in my code,
> and especially in understanding and finding bugs in other people’s

I mostly use prints and visual tracing, but I would probably be better 
off myself if IDLE's debugger were improved and I used it.  The IDLE doc 
still labels Debugger as "still incomplete and somewhat experimental". 
I would like to be able to delete that.

> code.  When I started using Python with IDLE, I was excited when I
> found that it had a debugger.  While the debugger has all the basic
> operations, I believe that the way you start the debugger  and the way
> that you set breakpoints in IDLE are highly unintuitive.  (In most
> other systems, breakpoints are set and cleared by clicking in a
> dedicated column next to the lines numbers, which turns on/off a
> simple dot or stop sign.)

Starting: I have thought that there should be Debug or Run with Debugger 
on the Run menu.  If a breakpoint is set, maybe make this automatic.

The rationale for only starting from the Shell might have been: 
"Starting debugger requires connection to a remote user process, which 
only exists if Shell is running."   Or maybe the current scheme is just 
a quick hack.  In either case, Run => Run Module starts Shell if it is 
not running and a run option could also start the debugger after 
starting the shell.

Breakpoints: How about clicking on the line number itself, once they are 
available?  I have though about changing the line number background to 
indicate breakpoint lines.

> I would really like to use the debugger in a class setting to
> demonstrate stepping through code.  For example, I’d like to
> demonstrate how argument values are passed and assigned to
> parameters, and also to demonstrate how control flows through a while
> loop with break and continue statements, etc.   I sometimes do this
> in class by copying a small program, and pasting it into
> PythonTutor.org, and run the code that way.  But it’s unfortunate
> that I cannot do this easily directly in IDLE.

I do not understand from the above what change you would like.

> The current debugger also has a flaw that makes it unworkable in a
> teaching environment.  Maybe I’m missing something, but the debugger
> seemingly cannot distinguish between user-written code and library
> code.

The crucial distinction is between 'code I want to trace' and 'other 
code'.  User have to indicate that somehow.  The current APi is [step] 
versus [go], [over], and [out] to skip.  Library code is someone's 
'user-written code'.  And as you say above, debuggers are especially 
useful for examining *other* people's code.  That said, Debugger *could* 
also pay attention to file paths.

 > As an example, if my code is at a breakpoint and is about to
> execute a print statement, and I click on “Step" to step through it,
> the debugger steps into the library code of the print statement

I now regard this as a bug.  See https://bugs.python.org/issue15335. 
But note that autoskipping Python code is a contentious idea.  A 
previous maintainer wrote "I fail to see the problem. Stepping through 
the standard library may be useful, and if you don't want to do that, 
you can choose to step over still."

> itself.  While I understand what is going on here, it would be
> impossible to explain this behavior to students who not only are new
> to debuggers, but new to programming.  Ideally, the Step button
> should only step a single line of the user’s code.  (PyCharm has this
> feature built into its debugger.)

Leaving PyCharm aside, what API might you like to see?  'Run with 
Debugger could pop up a box with

Restrict stepping to
[ ] Current file
[ ] Directory of current file
[ ] Directory ______________________
[x] Any Python file

> Beginning students see the Python runtime as a black box.  I do
> explain that the standard way of finding out what is going wrong with
> their programs by adding print statements.  However, students find
> that approach to be time consuming, and the output can be confusing
> as it may not be clear how the code got to certain print statements.

Interesting observation.  I have long forgotten whatever learning curve 
I went through to become expert at this.

> Instead, students typically wind up just making small changes to
> their code or adding new code, and look at how the results change.  I
> would love to explain how to my students how to use a debugger to
> step through code and watch the values of their variables change.
> But in its current form don’t even try to use the IDLE debugger in
> class.
>
> I have read through the documentation of recent open issues, but I
> haven’t seen anyone making any comments about the debugger.

Here is my list of open bugs.python.org IDLE debugger issues.
-Debugger
17942 Improve gui
15335 Steps over Idle rpc code, but into Idle print code
22083 Refactor breakpoint methods
14111 Handle interrupts
15347 Debugger activation disrupts closing even after debugger closed 
(Close)
15348*Closing [x] while active freezes shell (Serwy x 2)
24455 Closing twice while running caused crash.
24090 Send name value, possible truncated, to clipboard
24818 Isolate Shell connection so can run file without shell already open
25254 Highlight in source window again not working.
25146 Program visualization with intelligent auto step.
26949 raise SystemExit while debug causes extra Restart Shell

Feel free to post comments on the tracker (or here).

Here are some personal notes on other possible changes.

xxxxx Inspect attributes of global/local object, possible rt. click?
zzzzz At each step, color changed values.
nnnnn Suppress dunder names in globals list, so starts empty.
ppppp locals/globals min space for vars and space between longest and value
ddddd break on data change since monitor anyway?
mmmmm No Breakpoint setting on context menu?
https://python4kids.brendanscott.com/2015/09/02/python-for-kids-book-project-6/

 > As a
> teacher, having an easy-to-use debugger in IDLE would tremendously
> beneficial in student’s ability to understand and debug Python code.
>
> Maybe this should be a separate issue, but it would also be a great
> help if the lines of a program in the IDLE editor had line numbers

https://bugs.python.org/issue17535

> next to them.  It would make teaching concepts much easier to say,
> "look at lines 14 and 15 of your program", rather than pointing at
> them with my fat fingers to draw attention to where a student made a

This is pretty convincing.  I need to review the current patch and 
perhaps accept something less than 'perfect'.

> mistake.  It would also be much easier for students to identify the
> line numbers that appear in Python error messages, rather than having
> to click around in their source code and watch the line number change
> in the bottom right hand corner.

Are you aware, and do you teach students, that one can right click on a 
traceback line and jump to the file and line indicated?  Ditto for grep 
(Find in Files) output.  This is why I have not so strongly felt the 
need for line numbers myself.

 > Having a combination of line
> numbers AND a simpler debugger interface would be ideal!

Again, thanks for the feedback.  I agree and hope to implement both.

-- 
Terry Jan Reedy