reST Wiki mode, was [Doc-SIG] Suggestions for reST "modes"

[Garth T Kidd]

Interesting thinking on how to distinguish pragma and command
directives!

To minimise the impact on reversability, my current take is that pragma
directives (at minimum) have to be represented in the output as
directive tags, and -- if they hook the parser to extend what it can
output -- should contain elements describing what extended elements can
now be produced.

That way, an OOB parser can at *minimum* say "this element was loaded by
the Fnord extension, but not having the Fnord extension loaded (or it
not supporting reversability) I can't reproduce its original input".

"Simple" command directives like ``sidebar::``, where the directive
block is parsed and inserted under a directive of a the same name as the
extension, could be documented as such in the declaration of which
extensions are used in the document, in which case the parser can always
reproduce the original input.

> myself - if people are writing their own directive understanders, then
> we can't really mandate that they should provide the reverse
> mechanism.

No, but we should make sure that at least the parser can figure out
which extension was responsible for the unknown element, and complain
appropriately when trying to re-generate the input.

Any takers on a standard method of embedding the original input? Maybe
all non-simple directive-driven output *should* be resident in a
directive tag::

    <document>
      <extension name="grapher" type="complex">
        <exensiondirective name="graph"/>
      </extension>
      <extension name="sidebar" type="simple"/>

      <!-- ... -->

      <sidebar argument="">
        <paragraph>
          Simple extensions have no code associated with them, and
          implicitly declare a single directive of the same name.
          Such directives generate an output element of the extension
          name with the argument attribute as per generic directive
          tags, and contain pretty much any valid reST markup.
        </paragraph>
        <paragraph>
          Reversability is simple: if you see a tag named for a
          previously declared simple extension, reverse its
          contents and embed them in an appropriate directive.
        </paragraph>

      <!-- ... -->

      <directive type="graph">
        <originalinput>
           line 1,2 to 35,2
           line 1,6 to 430, 1
           ...
        </originalinput>
        <parsedoutput>
           <line>
              <coord x="1" y="2"/>
              <coord x="35" y="2"/>
           </line>
           <line>
              <coord x="1" y="6"/>
              <coord x="430" y="1"/>
           </line>

           <!-- ... -->

        </parsedoutput>
      </directive>

      <!-- ... -->

    </document>

> It does suggest to me that pragma directives must insert a node in the
> DOM tree, though, so that one can *know* what engines one needs to use
> to regenerate reST.

Amen.

>> .. _OOB: Out of the box, ie without any directives loaded.
>>     Now, how on Earth do we render this link target, or any
>>     other non-URI link target that hasn't been rescued by
>>     some directive? I'm using it as a footnote. Hmmm.
>
> Presumably one should not lose the footnote - it is still text, after
> all. Emit it with a warning that it is not referenced?

Do you see any square brackets? :)

Actually, it's not in our interest to try and resolve links -- that's
the application's problem, which is why we don't need to extend the
parser to handle WikiLinks or InterWikiLinks. We should, however, define
whether or not footnotes and normal links share the same space, etc.

I'm currently thinking that a link to a non-URI (and otherwise unres...

Damn.

* Let's say we want to use non-URI link targets as automatically
  numbered footnotes, which I think is pretty nifty. Explicitly
  named footnotes are still perfect_ for citing references, but
  authors are freed from the tyranny of having to avoid footnote
  number collisions.

  .. perfect: Perfection is great! And, since this link target
     doesn't have anything resembling a URI in it, it's an auto
     numbered footnote, right?

     How would anyone else make a non-footnote link to it? I
     have no idea. That said, I'm happy to make that particular
     rare case harder or impossible if it frees me from the need
     to make up unique identifiers for every footnote. I use a
     *lot* of footnotes. I'm sure David still has nightmares
     about my first few messages to this list. :)

  .. Yes, there's a loophole. Read on.

* If ``perfect_`` above is to become a footnote, the parser needs
  to know before it finishes its work that it needs to spit out a
  footnote tag, not a link target tag.

* In turn, that means we can't wait until the parser finishes for
  the application to parse the link target and figure out whether
  it resolves or not.

* Therefore, we need a way for applications to specify a resolver
  function to the parser. The function would be called when the
  parser can't make sense of a target, i.e. when::

  * The target text is not a URI, *and*

  * The target text itself doesn't parse as a link.

    If a target turns out to be a reST link, ie it's a
    single word or backquoted string followed by an underscore,
    the parser should treat it as an indirect link. That would
    let me refer to a section called ``Extremely Long Heading``
    as ``ELH_`` as follows::

      See ELH_ for more details.

      .. _ELH: `Extremely Long Heading`_

    That would otherwise be a footnote, but a pretty boring one. :)

* The edge case is footnotes that accidentally resemble something
  that the external resolver recognises. I think that's probably
  just too bad.

In summary:

+ The application should provide the parser with a function to
  handle resolution of links the parser can't otherwise figure
  out;

  (Note that this is a complete reversal of my previous position
  on link resolution :)

+ If the external link resolver can't figure it out, reST should
  treat it as an automatically numbered footnote; and

+ I'm still going on about indirect links, even though I suspect
  that they're already in the spec. I lack the time right now to
  wade through and check. Bueller?

Regards,
Garth.