[Doc-SIG] Clarification: interpreted text vs. directives vs. substitutions

[David Goodger]

I think there may be a fundamental misunderstanding as to the intended
function of interpreted text, directives, and substitutions. Easy to
understand, since the spec is lacking in explanation. I will update
the spec based on these discussions.

"Interpreted text" is text that is meant to be related, indexed,
linked, summarized, or otherwise processed, but the text itself is
left alone. The text is "tagged" directly, in-place. Interpreted text
may become a hyperlink reference, or may be character-formatted, or
may become an entry in an index, but the words within the backquotes
in the source text will not be altered. The "role" determines how the
text is interpreted. Roles are simply extensions of the available
inline constructs; to emphasis, strong, literal, and reference, we can
add "index entry" or "acronym" or "class" or "red" or "blinking" or
anything else we want.

Originally, I intended interpreted text to do some of what
substitutions now do (I had mentioned a ``:graphic:`picture.png```
example at some point). I think that was a mistake. By adding
substitutions, the meaning of interpreted text becomes clear.

The "role" should be inferred from context whenever possible. Using
the eventual Python source "Reader" component, the role of each bit of
interpreted text can be entirely inferred from the namespace of the
context::

    class B(A):
        """Use the `run()` method to do the work."""
        def __init__(self):
            """Set `self.a`."""
            self.a = 1
        def run(self):
            """Extend `A.run()`. `A` is an abstract base class."""
            pass

In the "B" class docstring, the reference to "run()" gets a role of
"method". In the "B.__init__()" docstring, "self.a" is an
"instance_attribute". In the "B.run()" docstring, "A.run()" is a
"method" and "A" is a "class". At each point in the definition, the
role is inferred from the local namespace, an analog to the way Python
itself does namespace lookup. The processed docstrings might represent
these as links to their definitions.

Another possible application is for index entries::

    Attach the `gizmo`:index: to the `widget`:index: with the
    `whatsit`:index:.

After processing, the interpreted text itself would be unchanged::

    Attach the gizmo to the widget with the whatsit.

But an index may have been generated by the processing system.

(Admittedly, the ":index:" suffixes are an eyesore. The role could
simply be ":x:", but that's not much better. If there's a better idea
out there, I'm all ears.)

"Directives" are meant for the arbitrary processing of their contents
(the directive data & text block), which can be transformed into
something possibly unrelated to the original text. Directives are a
constructive extension mechanism. They're for arbitrary structures
where there is no built-in syntax. Directives typically produce
structural or block-level (body) elements, but they can also affect
processing in other ways (not yet explored). Example::

    .. figure:: picture.png

       This is the caption of the figure (a simple paragraph).

       +-----------------------+-----------------------+
       | Symbol                | Meaning               |
       +=======================+=======================+
       | .. image:: tent.png   | Campground            |
       +-----------------------+-----------------------+
       | .. image:: waves.png  | Lake                  |
       +-----------------------+-----------------------+
       | .. image:: peak.png   | Mountain              |
       +-----------------------+-----------------------+

"Substitutions" are a text-level (inline) construct, a way of getting
a directive into the middle of some text. The substitution reference
encloses a name, which must match with the name on the substitution
itself. The substitution itself is a named, indirect directive, which
must produce an inline-compatible object (text or inline elements)::

    The `/biohazard/` symbol must be used on containers used to
    dispose of medical waste.

    .. /biohazard/ image:: biohazard.png

Note that I've dropped the "replacement text" aspect of substitutions.
With this change, "substitution references" could be renamed "indirect
inline directive references" (or "inline directive references" or
"directive references" for short). "Substitutions" could be renamed
"indirect directives" or "indirect directive definitions".

Again, I'm open to any better ideas for syntax out there. Lay 'em on me!

-- 
David Goodger    goodger@users.sourceforge.net    Open-source projects:
 - Python Docstring Processing System: http://docstring.sourceforge.net
 - reStructuredText: http://structuredtext.sourceforge.net
 - The Go Tools Project: http://gotools.sourceforge.net