[Doc-SIG] Comments on the reST specification - comments

[Garth T Kidd]

>>> You misunderstand. I'm talking about comments only, not directives.
>>
>> Directives will chew everything until an outdent, but
>> comments won't?
>>
>> I'm obviously still misunderstanding.
>
> I'd like to keep everything orthogonal, if possible. One-liner
> comments seem less attractive to me today, so it's probably
> "comment" directives or the status quo. Give me a few days to
> mull it over.

Status quo requires less code change. :)

> In the meantime, can you come up with a way to assuage Tony's concerns
> regarding simple ``.. `` comment starts? Specifically:
>
> 1. The inability to start a comment with ``_target text:`` or
>    ``directive::``.

Tony also said at first he couldn't imagine using comments at all. :)

This is another one of those "make the common case difficult to make a
rare case easier" tradeoffs. As usual, my position is that we should
keep the common case easier unless the rare case becomes downright
impossible.

Any experienced reST author will know that anything beginning ``.. _``
will be interpreted as a link and that a comment starting with a
single-word identifier followed by a double colon will be treated as a
directive.

The workarounds are, to me, simple:

* My top-of-mind reason to start a comment with an underscore is when
  referring to an identifier that happens to start with an underscore.
  In my body text, I'd be using ```target``` or ````target```` when
  referring to identifiers, so the obvious way to refer to an
  underscored identifier in a comment would be::

    .. `_hide_me` is a messy workaround, but backquotes aren't.

* Similarly, if I'm referring to a directive::

    .. ``comment::`` is dingo-ugly, and should be spurned.

* If I needed to go another level meta, I'd dodge the extra set of
  backquotes and use a literal block::

    .. ::

         ``comment``:: is dingo-ugly

       is a bit harsh.

Comments are, of course, passed to the output as an unparsed block of
text (no paragraphs, no literal blocks, etc), but the important factors
have been preserved:

* anyone reading the document should find it pretty obvious what's
  going on, and

* the techniques should be intuitive to experienced reST authors.

In the IDG Books' "reStructuredText Markup for Dummies", using literals
to make sure your comments are parsed as comments would be a
single-paragraph "Remember" or "Tip". It's not a problem worth the
hassle of making everyone type ``.. comment::` for every comment. I
don't think it's even worth the bother of making an optional
``comment::`` directive -- the literals are quicker to type, so I'd
certainly be using literals if given an option.

> 2. The concern about future comment-breaking (!), if we ever introduce
>    another explicit markup construct. (I don't foresee the need now,
>    but that's to be expected.)

I see a potential problem with directive arguments. Maybe it's just the
overload on the operastor, but to me whatever comes after the ``::`` is
a literal argument to the directive. If the directive has multiple
paragraphs, my mental model says they're all parsed as a literal block
(sans the directive marker).

The parser says different::

    .. sidebar:: this is some text

       this is a second para

comes out as::

    <document>
        <directive data="this is some text" type="sidebar">
            <literal_block>
                this is a second para
            </literal_block>
        </directive>
    </document>

not as I expect, which would be::

    <document>
        <directive type="sidebar">
            <literal_block>
                this is some text

                this is a second para
            </literal_block>
        </directive>
    </document>

... which I could then. I think my mental model runs that way because it
is slightly more consistent (to my poor addled brain, at least) with the
comment equivalent, for which::

    .. this is some text

       this is a second para

comes out as::

    <document>
        <comment>
            this is some text

            this is a second para
        </comment>
    </document>

To bring directives, comments, and explicit directive-driven comments in
line with each other, I think we should:

* Retain ``.. [anything which doesn't parse as a link or directive`` as
  being implicitly equivalent to ``.. comment:: [anything...``

  [Actually, let's not. See below.]

* Add a ``comment::`` directive if people can't bear making something
  consistent with a directive that doesn't actually exist. :)

* Remove the literal block node from the directive.

That way I can revisit a multiple-paragraph implicit comment directive
and convert it into a sidebar by inserting ``sidebar:: `` at the
beginning of the first line in the first paragraph of the comment. Very
clean.

The question then becomes:

  What happens if I want to say "sidebar=right" or "sidebar=left"?

::

  .. sidebar side=left:: text...
  .. sidebar left:: text...
  .. sidebar(side=left):: text...
  .. sidebar(side="left"):: text...
  .. ...

Ewwww. Ugly. The reason for ``data=`` abruptly becomes clear. Good call,
David. :) I'll settle for inserting ``sidebar::`` and a couple of
newlines (or, it turns out, just one -- see below) as a solution for my
"convert comment to sidebar" case.

>>> No, an empty comment *will* add a node to the tree.

>> My imagination must be cramped. Why?

> I want the output to have a one-to-one correspondence to the
> input, so that round trips are possible (without cheating by
> looking at the "rawsource" attribute).

Good call.

> I indend to have an output formatter which produces reStructuredTest.

Yaay!

>>> So we special-case the empty comment (== two dots &
>>> [optional] space *only* on the first line?) to *not* consume
>>> subsequent indented blocks?
>>
>> Yes. Exactly.
>
> OK, interesting idea. What about an empty comment start followed by an
> indented non-blank line? ::
>
>     ..
>        Is this part of the comment or not?

According to the parser, yes. :)

.. My girlfriend hates it when I dodge like that, but the joke's on me.
   I've been tapping away on this message for so long the food hall
   downstairs will have almost nothing left by now.

I agree with the parser. There being no outdent, that's part of the
comment, for sure, which means my empty comment special case needs a
blank line after it.

I'm more worried about how the parser handles::

    $ ./quicktest.py
    .. directive::
       is it?
    =>
    <document>
        <directive type="directive">
            <literal_block>
                is it?
            </literal_block>
        </directive>
    </document>

... which places a natural limit on just how directive arguments can get
if the author is unfortunate enough to have a crummy text editor. The
most obvious workaround doesn't::

    $ ./quicktest.py
    .. directive:: blah\
       blah blah
       is it?
    =>
    <document>
        <directive data="blah\" type="directive">
            <literal_block>
                blah blah
                is it?
            </literal_block>
        </directive>
    </document>

Hmmm. ::

    $ ./quicktest.py
    this\
    that
    =>
    <document>
        <paragraph>
            this
            that
        </paragraph>
    </document>

Okay, that explains it. Maybe I just made up that bit about using
backslashes for line continuation. Worth adding to the spec, or is that
another accident waiting to happen?

Regards,
Garth.