[Python-Dev] The docstring hack for signature information has to go

[Terry Reedy]

On 2/3/2014 9:43 AM, Larry Hastings wrote:
>
>
> A quick summary of the context: currently in CPython 3.4, a builtin
> function can publish its "signature" as a specially encoded line at the
> top of its docstring.  CPython internally detects this line inside
> PyCFunctionObject.__doc__ and skips past it, and there's a new getter at
> PyCFunctionObject.__text_signature__ that returns just this line.  As an
> example, the signature for os.stat looks like this:
>
>      sig=($module, path, *, dir_fd=None, follow_symlinks=True)
>
> The convention is, if you have this signature, you shouldn't have your
> docstring start with a handwritten signature like 3.3 and before.
> help() on a callable displays the signature automatically if it can, so
> if you *also* had a handwritten signature, help() would show two
> signatures.  That would look dumb.
>
> -----
>
> So here's the problem.  Let's say you want to write an extension that
> will work with Python 3.3 and 3.4, using the stable ABI.  If you don't
> add this line, then in 3.4 you won't have introspection information,
> drat.  But if you *do* add this line, your docstring will look mildly
> stupid in 3.3, because it'll have this unsightly "sig=(" line at the
> top.  And it *won't* have a nice handwritten docstring.  (And if you
> added both a sig= signature *and* a handwritten signature, in 3.4 it
> would display both.  That would also look dumb.)

I think the solution adopted should be future-oriented (ie, clean in the 
future) even if the cost is slight awkwardness in 3.3. To me, an 
temporary 'unsightly' extra 'sig=' at the start of some docstrings, in 
one release, is better that any of the permanent contortions you propose 
to avoid it. For 3.3.5 Idle, I could add a check to detect and remove 
'sig=' from calltips, but I would not consider it a disaster for it to 
appear with earlier versions. In 3.3.5 (assuming no change is possible 
for 3.3.4), help (or pydoc) could make the same check and deletion. As 
with calltips, help is for interactive viewing by humans.

[snip]

> O: Have the handwritten signature in the docstring.  When registering
> the function for 3.3, have the PyMethodDef docstring point to the it
> starting at the signature.  When registering the function for 3.4+, have
> the docstring in the PyMethodDefEx point to the first byte after the
> handwritten signature.  Note that automatically skipping the signature
> with a heuristic is mildly complicated, so this may be hard to get right.

The old convention builtins was a one line handwritten signature 
followed by a blank line. For Python functions, one line describing the 
return value. The 'heuristic' for Idle was to grab the first line of the 
docstring. If than ended in mid-sentence because someone did not follow 
the convention, too bad.

The newer convention for builtins is multiple lines followed by a blank 
line. So I recently changed the heuristic to all lines up to the first 
blank, but with a limit of 5 (needed for bytes), as protection against 
doctrings that start with a long paragraph.

-- 
Terry Jan Reedy