[PEP 224] Attribute Docstrings

[M.-A. Lemburg]

An HTML version of the attached can be viewed at

    http://python.sourceforge.net/peps/pep-0224.html

Even though the implementation won't go into Python 2.0, it
is worthwhile discussing this now, since adding these attribute
docstrings to existing code already works: Python simply ignores
them. What remains is figuring out a way to make use of them and
this is what the proposal is all about...

--

PEP: 224
Title: Attribute Docstrings
Version: $Revision: 1.2 $
Author: mal at lemburg.com (Marc-Andre Lemburg)
Status: Draft
Type: Standards Track
Python-Version: 2.1
Created: 23-Aug-2000
Post-History:


Introduction

    This PEP describes the "attribute docstring" proposal for Python
    2.0.  This PEP tracks the status and ownership of this feature.
    It contains a description of the feature and outlines changes
    necessary to support the feature.  The CVS revision history of
    this file contains the definitive historical record.


Rationale

    This PEP proposes a small addition to the way Python currently
    handles docstrings embedded in Python code.

    Python currently only handles the case of docstrings which appear
    directly after a class definition, a function definition or as
    first string literal in a module.  The string literals are added
    to the objects in question under the __doc__ attribute and are
    from then on available for introspection tools which can extract
    the contained information for help, debugging and documentation
    purposes.

    Docstrings appearing in locations other than the ones mentioned
    are simply ignored and don't result in any code generation.

    Here is an example:

        class C:
            "class C doc-string"

            a = 1
            "attribute C.a doc-string (1)"

            b = 2
            "attribute C.b doc-string (2)"

    The docstrings (1) and (2) are currently being ignored by the
    Python byte code compiler, but could obviously be put to good use
    for documenting the named assignments that precede them.
    
    This PEP proposes to also make use of these cases by proposing
    semantics for adding their content to the objects in which they
    appear under new generated attribute names.

    The original idea behind this approach which also inspired the
    above example was to enable inline documentation of class
    attributes, which can currently only be documented in the class's
    docstring or using comments which are not available for
    introspection.


Implementation

    Docstrings are handled by the byte code compiler as expressions.
    The current implementation special cases the few locations
    mentioned above to make use of these expressions, but otherwise
    ignores the strings completely.

    To enable use of these docstrings for documenting named
    assignments (which is the natural way of defining e.g. class
    attributes), the compiler will have to keep track of the last
    assigned name and then use this name to assign the content of the
    docstring to an attribute of the containing object by means of
    storing it in as a constant which is then added to the object's
    namespace during object construction time.

    In order to preserve features like inheritance and hiding of
    Python's special attributes (ones with leading and trailing double
    underscores), a special name mangling has to be applied which
    uniquely identifies the docstring as belonging to the name
    assignment and allows finding the docstring later on by inspecting
    the namespace.

    The following name mangling scheme achieves all of the above:

        __doc__<attributename>__

    To keep track of the last assigned name, the byte code compiler
    stores this name in a variable of the compiling structure.  This
    variable defaults to NULL.  When it sees a docstring, it then
    checks the variable and uses the name as basis for the above name
    mangling to produce an implicit assignment of the docstring to the
    mangled name.  It then resets the variable to NULL to avoid
    duplicate assignments.

    If the variable does not point to a name (i.e. is NULL), no
    assignments are made.  These will continue to be ignored like
    before.  All classical docstrings fall under this case, so no
    duplicate assignments are done.

    In the above example this would result in the following new class
    attributes to be created:

        C.__doc__a__ == "attribute C.a doc-string (1)"
        C.__doc__b__ == "attribute C.b doc-string (2)"

    A patch to the current CVS version of Python 2.0 which implements
    the above is available on SourceForge at [1].


Caveats of the Implementation
    
    Since the implementation does not reset the compiling structure
    variable when processing a non-expression, e.g. a function
    definition, the last assigned name remains active until either the
    next assignment or the next occurrence of a docstring.

    This can lead to cases where the docstring and assignment may be
    separated by other expressions:

        class C:
            "C doc string"

            b = 2

            def x(self):
                "C.x doc string"
                y = 3
                return 1

            "b's doc string"

    Since the definition of method "x" currently does not reset the
    used assignment name variable, it is still valid when the compiler
    reaches the docstring "b's doc string" and thus assigns the string
    to __doc__b__.

    A possible solution to this problem would be resetting the name
    variable for all non-expression nodes.

    
Copyright

    This document has been placed in the Public Domain.


References

    [1] http://sourceforge.net/patch/?func=detailpatch&patch_id=101264&group_id=5470

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/