[Python-Dev] The Return Of Argument Clinic

[Nick Coghlan]

On 6 August 2013 09:53, Larry Hastings <larry at hastings.org> wrote:
> On 08/05/2013 02:55 AM, Nick Coghlan wrote:
> On 5 August 2013 18:48, Larry Hastings <larry at hastings.org> wrote:
>
> Question 0: How should we integrate Clinic into the build process?
>
> Isn't solving the bootstrapping problem the reason for checking in the
> clinic-generated output? If there's no Python available, we build what
> we have (without the clinic step), then we build it again *with* the
> clinic step.
>
> It solves the bootstrapping problem, but that's not the only problem Clinic
> presents to the development workflow.
>
> If you modify some Clinic DSL in a C file in the CPython tree, then run
> "make", should the Makefile re-run Clinic over that file?  If you say "no",
> then there's no problem.  If you say "yes", then we have the problem I
> described.

Ah, I think I see the problem you mean. What is defined in the
makefile as the way to regenerate an object file from the C file. If
it is run clinic and then run the compiler, then you will get a
dependency loop. If it doesn't implicitly run clinic, then we risk
checking in inconsistent clinic metadata.

I think the simplest answer may be to have "make clinic" as an
explicit command, along with a commit hook that checks for clinic
metadata consistency. Then "make" doesn't have to change and there's
no nasty bootstrapping problem.


> ___________________________________________________________________
> Question 2: Emit code for modules and classes?
>
> There are some complications to this, one of which I'll
> discuss next.  But I put it to you, gentle reader: how
> much boilerplate should Argument Clinic undertake to
> generate, and how much more class and module metadata
> should be wired in to it?
>
> I strongly recommend deferring this. Incremental development is good,
> and getting this bootstrapped at all is going to be challenging enough
> without trying to do everything at once.
>
>
> I basically agree.  But you glossed over an important part of that question,
> "how much more class and module metadata should be wired in right now?".
>
> Originally Clinic didn't ask for full class and module information, you just
> specified the full dotted path and that was that.  But that's ambiguous;
> Clinic wouldn't be able to infer what was a module vs what was a class.  And
> in the future, if/when it generates module and class boilerplate, obviously
> it'll need to know the distinction.  I figure, specifying the classes and
> modules doesn't add a lot of additional cost, but it'll very likely save us
> a lot of time in the long run, so I made it a requirement.  (WAGNI!)

Note that setuptools entry point syntax solves the namespace ambiguity
problem by using ":" to separate the module name from the object's
name within the module (the nost test runner does the same thing). I'm
adopting that convention for the PEP 426 metadata, and it's probably
appropriate as a concise notation for clinic as well.

> As long as the code Clinic generates is backwards compatible for Python 3.4,
> I think this will has it covered.  We may at times force developers to use
> fresher versions of Python to process Clinic stuff, but I don't think that's
> a big deal.

One of the nice things about explicitly versioned standards is that
you can set the "no version stated" to the first version released and
avoid the boilerplate in the common case :)

> ___________________________________________________________________
> Question 5: Keep too-magical class decorator Converter.wrap?
> Let's say I go with your proposal.  What happens if someone makes a
> Converter, and wraps it with Converter.wrap, and defines their own __init__?
> It would never get called.  Silently, by default, which is worse--though I
> could explicitly detect such an __init__ and throw an exception I guess.
> Still, now we have a class where you can't use the name __init__, you have
> to use this funny other name, for arbitrary "correctness" reasons.

You misunderstand me: I believe a class decorator is the *wrong
solution*. I am saying Converter.wrap *shouldn't exist*, and that the
logic for what it does should be directly in Converter.__init__.

The additional initialisation method could be given a better name like
"process_custom_params" rather than "custom_init".

That is, instead of this hard to follow magic:

    @staticmethod
    def wrap(cls):
        class WrappedConverter(cls, Converter):
            def __init__(self, name, function, default=unspecified,
                *, doc_default=None, required=False, **kwargs):
                super(cls, self).__init__(name, function, default,
                    doc_default=doc_default, required=required)
                cls.__init__(self, **kwargs)
        return functools.update_wrapper(
WrappedConverter,
            cls, updated=())

You would just have the simple:

    class Converter:
        def __init__(self, name, function, default=unspecified,
                *, doc_default=None, required=False, **kwargs):
            .... # Existing arg process
            self.process_custom_params(**kwargs)

        def process_custom_params(self):
            # Default to no custom parameters allowed
            pass

Those that just want to define custom parameters and leave the rest of
the logic alone can override "process_custom_params". Those that want
to completely control the initialisation can override __init__
directly.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia