[Python-Dev] [Python-checkins] peps: New DSL syntax and slightly changed semantics for the Argument Clinic DSL.

[Larry Hastings]

On 03/17/2013 03:26 PM, Stefan Krah wrote:
> While I like the syntax better and appreciate the option to condense the
> function declaration I still fear that the amount of implicitness will
> distract from what is important: programming in C.
>
> This applies especially if people start declaring converters using the
> [python] feature.
>
> So I hope that at least converters can be declared statically in a header
> file, like I suggested in PEP 437.

The Argument Clinic prototype is written in Python; I don't know how 
"declared static in a header file" applies to a Python implementation.  
Currently the converters are declared directly in clinic.py, somewhere 
in the middle.

For what it's worth, I think the new syntax (dictated more-or-less by 
Guido and Nick) makes things slightly less explicit.  In my original 
syntax, you declared the *type* of each parameter in C, and the flags 
dictated how Clinic should perform the conversion.  In the new syntax, 
you declare the *converter function* for each parameter, and the type of 
the resulting C variable is inferred.


> A couple of comments:
>> As of CPython 3.3, builtin functions nearly always parse their arguments
>> with one of two functions: the original ``PyArg_ParseTuple()``, [1]_ and
>> the more modern ``PyArg_ParseTupleAndKeywords()``. [2]_ The former
>> only handles positional parameters; the latter also accommodates keyword
>> and keyword-only parameters, and is preferred for new code.
> What is the source for this? I seem to remember a discussion on python-ideas
> (but cannot find it now) where some developers preferred non-keyword functions
> for some use cases.
>
> For example it's strange to write div(x=10, y=3), or worse, div(y=3, x=10).
> Using positional-only arguments prevents this "feature".

(I don't know to what "div" function you refer, but its arguments are 
poorly named.)

I thought this was obviously true.  But realized I didn't have any 
justification for it.  So I checked into it.  Thomas Wouters found me a 
thread from python-ideas from last March about changing range() to 
support keyword-only parameters.  Guido participated in the thread, and 
responded with this pronouncement:

    [...] let's forget about "fixing" range. And that's final.

    http://mail.python.org/pipermail/python-ideas/2012-March/014380.html

I double-checked with him about this and he confirmed: positional 
parameters are here to stay.

This has some consequences.  For example, inspect.getfullargspec, 
inspect.Signature, and indeed types.FunctionObject and types.CodeObject 
have no currently defined mechanism for communicating that a parameter 
is positional-only.  I strongly assert we need such a mechanism, though 
it could be as simple as having the parameter name be an empty string or 
None.

Anyway, it turns out this "keyword-only is preferred" was a 
misconception on my part.  Python supports, and will continue to 
support, positional-only parameters as part of the language. Currently 
it isn't possible to define functions in Python that have them.  But 
builtins have them, and will continue to have them, and therefore 
Argument Clinic needs to support them.

I'll amend my PEP soonish to reflect this.  Specifically the semantics 
of the /, [, and ] lines in the parameter section.


>>     path: path_t(allow_fd=1)
>>
>> I do not see where the C initialization or the cleanup are specified. Are
>> they part of the converter specification?

The extension interface isn't yet well-enough defined to be in the PEP.  
But yes, the generation of cleanup code will be part of the job of the 
conversion functions.  I'm not sure I have any provision for 
initialization apart from assignment, but I agree that it should have one.


>> curses.window.addch
> The parameters appear to be in the wrong order.

Fixed.


>> The return annotation is also optional.  If skipped, the arrow ("``->``")
>> must also be omitted.
> Why is it optional? Aren't type annotations important?

I'm not aware of any builtins that have annotations.  In fact, I'm not 
sure there's any place in the API where you can attach annotations to a 
builtin.  I expect this will change when we add reflection support for 
signatures, though I doubt many builtins will use the facility.

This raises an interesting point, though: Argument Clinic provides a 
place to provide a function return annotation, but has never supported 
per-parameter annotations.  I'll meditate on this and see if I can come 
up with a reasonable amendment to the current syntax.


> How are the converters specified? Inside the preprocessor source? Are initialization
> and cleanup part of the specification, e.g. is a converter represented by a class?

Respectively: in Python, yes, and yes.  The current prototype has an 
experimental extension interface; this is used to add support for 
"path_t" in Modules/posixmodule.c.  It supports initialization and cleanup.


> I think there should be a table that lists which values are converted and what
> the result of the conversion is.

The documentation is TBD.  In the current prototype, it explicitly turns 
"None" into "Py_None", and otherwise flushes whatever you specified as 
the default value through from the DSL to the generated C code.


>> types
>>
>>     A list of strings representing acceptable Python types for this object.
>>     There are also four strings which represent Python protocols:
> I don't quite follow: Aren't input types always specified by the converter
> function?

I'm not sure yet.  The purpose of parameterizing the converter functions 
is to cut down on having eleventy-billion (in other words, "forty") 
different converter functions, some with only slight differences.  For 
example, 'C' accepts a str of length 1, whereas 'c' accepts a bytes or 
bytearray of length 1.  Should this be two converter functions, or one 
converter taking a list of types?  I'll see what feels better.


> The example in posixmodule.c takes up a lot of space and from the perspective
> of auditing the effects it's a little like following a longjmp.

I got strong feedback that I needed more examples.  That was the logical 
place for them.  Can you suggest a better spot, or spots?


//arry/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130318/9eaaaccf/attachment.html>