module attributes and docstrings

[Steven D'Aprano]

On Tue, 24 Mar 2015 07:55 pm, Mario Figueiredo wrote:

> Reading PEP 257 and 258 I got the impression that I could document
> module attributes and these would be available in the __doc__
> attribute of the object.

PEP 258 is rejected, so you can't take that as definitive.

PEP 257 has this definition very early in the document:

    A docstring is a string literal that occurs as the first 
    statement in a module, function, class, or method definition.


Nothing there about documenting arbitrary attributes.

The PEP does go on to say:

    String literals occurring elsewhere in Python code may also 
    act as documentation. They are not recognized by the Python
    bytecode compiler and are not accessible as runtime object
    attributes (i.e. not assigned to  __doc__  ), but two types
    of extra docstrings may be extracted by software tools: ...

so if I write this:

class K:
    x = something()
    """Documentation for x"""

then the string *may* be extracted by certain third-party tools, but it will
not be recognised as a docstring by the Python interpreter and will not be
available as the __doc__ attribute of K.x.


> So things like the one below are something I got used to do, but that
> don't work after all, as I learned today:
> 
>     value_factory = lambda _, row: row[0]
>     """Row factory. To be used with single-column queries."""
> 
> There are other things I could possibly do, like turning that lambda
> into a function, or document attributes in the module docstring. They
> are fair responses.


For the record, the lambda is already a function. lambda is just an
alternative syntax for creating functions, not a different kind of object.


> But did I read docstring extraction rules in PEP 258 wrong by assuming
> that the above example would make into the __doc__ attribute? And if
> so, short of documenting attributes in the module docstring, is there
> another way I can document individual attributes?

In a word, no.

Even if there was support from the compiler to extract the docstring, where
would it be stored? Consider:

spam = None
"""Spammy goodness."""
eggs = None
"""Scrambled, not fried."""

There's only one None object, and even if it could take a docstring (and it
can't), which docstring would it get? Presumably the second, which would
make help(spam) confusing, but when we say eggs = 23 the docstring would
disappear too.

Python's data model is not compatible with the idea of associating
docstrings with arbitrary attributes or variables. The docstring has to be
associated with an object, and only certain objects at that.

That applies to documentation which is introspectable at runtime. But of
course you can do anything you like by parsing the source code! It may be
that the docutils library will parse the source code and extract
docstrings, associating them to their variable or attribute. If not,
perhaps some other third-party library, or you can write your own.



-- 
Steven