[Python-Dev] PEP 553, v3

Wed Sep 13 22:12:46 EDT 2017

Here’s an update to PEP 553, which makes $PYTHONBREAKPOINT a first class feature.  I’ve also updated PR #3355 with the implementation to match.

Cheers,
-Barry

PEP: 553
Title: Built-in breakpoint()
Author: Barry Warsaw <barry at python.org>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 2017-09-05
Python-Version: 3.7
Post-History: 2017-09-05, 2017-09-07, 2017-09-13

Abstract
========

This PEP proposes adding a new built-in function called ``breakpoint()`` which
enters a Python debugger at the point of the call.  Additionally, two new
names are added to the ``sys`` module to make the debugger pluggable.

Rationale
=========

Python has long had a great debugger in its standard library called ``pdb``.
Setting a break point is commonly written like this::

    foo()
    import pdb; pdb.set_trace()
    bar()

Thus after executing ``foo()`` and before executing ``bar()``, Python will
enter the debugger.  However this idiom has several disadvantages.

* It's a lot to type (27 characters).

* It's easy to typo.  The PEP author often mistypes this line, e.g. omitting
  the semicolon, or typing a dot instead of an underscore.

* It ties debugging directly to the choice of pdb.  There might be other
  debugging options, say if you're using an IDE or some other development
  environment.

* Python linters (e.g. flake8 [linters]_) complain about this line because it
  contains two statements.  Breaking the idiom up into two lines further
  complicates the use of the debugger,

These problems can be solved by modeling a solution based on prior art in
other languages, and utilizing a convention that already exists in Python.

Proposal
========

The JavaScript language provides a ``debugger`` statement [java]_ which enters
the debugger at the point where the statement appears.

This PEP proposes a new built-in function called ``breakpoint()``
which enters a Python debugger at the call site.  Thus the example
above would be written like so::

    foo()
    breakpoint()
    bar()

Further, this PEP proposes two new name bindings for the ``sys``
module, called ``sys.breakpointhook()`` and
``sys.__breakpointhook__``.  By default, ``sys.breakpointhook()``
implements the actual importing and entry into ``pdb.set_trace()``,
and it can be set to a different function to change the debugger that
``breakpoint()`` enters.  ``sys.__breakpointhook__`` then stashes the
default value of ``sys.breakpointhook()`` to make it easy to reset.
This exactly models the existing ``sys.displayhook()`` /
``sys.__displayhook__`` and ``sys.excepthook()`` /
``sys.__excepthook__`` [hooks]_.

The signature of the built-in is ``breakpoint(*args, **kws)``.  The positional
and keyword arguments are passed straight through to ``sys.breakpointhook()``
and the signatures must match or a ``TypeError`` will be raised.  The return
from ``sys.breakpointhook()`` is passed back up to, and returned from
``breakpoint()``.  Since ``sys.breakpointhook()`` by default calls
``pdb.set_trace()`` by default it accepts no arguments.

Environment variable
====================

The default implementation of ``sys.breakpointhook()`` consults a new
environment variable called ``PYTHONBREAKPOINT``.  This environment variable
can have various values:

* ``PYTHONBREAKPOINT=0`` disables debugging.  Specifically, with this value
  ``sys.breakpointhook()`` returns ``None`` immediately.

* ``PYTHONBREAKPOINT= `` (i.e. the empty string).  This is the same as not
  setting the environment variable at all, in which case ``pdb.set_trace()``
  is run as usual.

* ``PYTHONBREAKPOINT=some.importable.callable``.  In this case,
  ``sys.breakpointhook()`` imports the ``some.importable`` module and gets the
  ``callable`` object from the resulting module, which it then calls.  The
  value may be a string with no dots, in which case it names a built-in
  callable, e.g. ``PYTHONBREAKPOINT=int``.  (Guido has expressed the
  preference for normal Python dotted-paths, not setuptools-style entry point
  syntax [syntax]_.)

This environment variable allows external processes to control how breakpoints
are handled.  Some uses cases include:

* Completely disabling all accidental ``breakpoint()`` calls pushed to
  production.  This could be accomplished by setting ``PYTHONBREAKPOINT=0`` in
  the execution environment.  Another suggestion by reviewers of the PEP was
  to set ``PYTHONBREAKPOINT=sys.exit`` in this case.

* IDE integration with specialized debuggers for embedded execution.  The IDE
  would run the program in its debugging environment with ``PYTHONBREAKPOINT``
  set to their internal debugging hook.

``PYTHONBREAKPOINT`` is re-interpreted every time ``sys.breakpointhook()`` is
reached.  This allows processes to change its value during the execution of a
program and have ``breakpoint()`` respond to those changes.  It is not
considered a performance critical section since entering a debugger by
definition stops execution.  (Of note, the implementation fast-tracks the
``PYTHONBREAKPOINT=0`` case.)

Overriding ``sys.breakpointhook`` defeats the default consultation of
``PYTHONBREAKPOINT``.  It is up to the overriding code to consult
``PYTHONBREAKPOINT`` if they want.

If access to the ``PYTHONBREAKPOINT`` callable fails in any way (e.g. the
import fails, or the resulting module does not contain the callable), a
``RuntimeWarning`` is issued, and no breakpoint function is called.

Open issues
===========

Confirmation from other debugger vendors
----------------------------------------

We want to get confirmation from at least one alternative debugger
implementation (e.g. PyCharm) that the hooks provided in this PEP will
be useful to them.

Evaluation of $PYTHONBREAKPOINT
-------------------------------

There has been some mailing list discussion around this topic.  The basic
behavior as described above does not appear to be controversial.  Guido has
expressed a preference for ``$PYTHONBREAKPOINT`` consultation happening in
the default implementation of ``sys.breakpointhook`` [envar]_.

The one point of discussion relates to whether the value of
``$PYTHONBREAKPOINT`` should be loaded on interpreter start, and whether its
value should be cached the first time it's accessed.

It is the PEP author's opinion that the environment variable need only be
looked up at the time of use.  It is also the author's opinion that the value
of the environment variable can be accessed each time ``sys.breakpointhook``
is run, to allow for maximum functional flexibility.  Because this feature
enters the debugger, any performance improvements for caching will be
negligible and do not outweigh the flexibility.  Further, because the special
case of ``PYTHONBREAKPOINT=0`` is fast-tracked, the no-op code path is quite
fast, and should be in the noise given the function calls of ``breakpoint()``
-> ``sys.breakpointhook()``.

Breakpoint bytecode
-------------------

Related, there has been an idea to add a bytecode that calls
``sys.breakpointhook()``.  Whether built-in ``breakpoint()`` emits
this bytecode (or gets peephole optimized to the bytecode) is an open
issue.  The bytecode is useful for debuggers that actively modify
bytecode streams to trampoline into their own debugger.  Having a
"breakpoint" bytecode might allow them to avoid bytecode modification
in order to invoke this trampoline.  *NOTE*: It probably makes sense to split
this idea into a separate PEP.

Call a fancier object by default
--------------------------------

Some folks want to be able to use other ``pdb`` interfaces such as
``pdb.pm()``.  Although this is a less commonly used API, it could be
supported by binding ``sys.breakpointhook`` to an object that implements
``__call__()``.  Calling this object would call ``pdb.set_trace()``, but the
object could expose other methods, such as ``pdb.pm()``, making invocation of
it as handy as ``breakpoint.pm()``.

Implementation
==============

A pull request exists with the proposed implementation [impl]_.

Rejected alternatives
=====================

A new keyword
-------------

Originally, the author considered a new keyword, or an extension to an
existing keyword such as ``break here``.  This is rejected on several fronts.

* A brand new keyword would require a ``__future__`` to enable it since almost
  any new keyword could conflict with existing code.  This negates the ease
  with which you can enter the debugger.

* An extended keyword such as ``break here``, while more readable and not
  requiring a ``__future__`` would tie the keyword extension to this new
  feature, preventing more useful extensions such as those proposed in
  PEP 548.

* A new keyword would require a modified grammar and likely a new bytecode.
  Each of these makes the implementation more complex.  A new built-in breaks
  no existing code (since any existing module global would just shadow the
  built-in) and is quite easy to implement.

sys.breakpoint()
----------------

Why not ``sys.breakpoint()``?  Requiring an import to invoke the debugger is
explicitly rejected because ``sys`` is not imported in every module.  That
just requires more typing and would lead to::

    import sys; sys.breakpoint()

which inherits several of the problems this PEP aims to solve.

Version History
===============

* 2017-09-13

  * The ``PYTHONBREAKPOINT`` environment variable is made a first class
    feature.

* 2017-09-07

  * ``debug()`` renamed to ``breakpoint()``
  * Signature changed to ``breakpoint(*args, **kws)`` which is passed straight
    through to ``sys.breakpointhook()``.

References
==========

.. [linters]
   http://flake8.readthedocs.io/en/latest/

.. [java]
   https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/debugger

.. [hooks]
   https://docs.python.org/3/library/sys.html#sys.displayhook

.. [syntax]
    http://setuptools.readthedocs.io/en/latest/setuptools.html?highlight=console#automatic-script-creation

.. [impl]
   https://github.com/python/cpython/pull/3355

.. [envar]
   https://mail.python.org/pipermail/python-dev/2017-September/149447.html

Copyright
=========

This document has been placed in the public domain.

..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   coding: utf-8
   End:

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL: <http://mail.python.org/pipermail/python-dev/attachments/20170913/508a0911/attachment.sig>