[Python-Dev] Experiment an opt-in new C API for Python? (leave current API unchanged)

[Brett Cannon]

On Wed, 14 Nov 2018 at 16:09, Gregory P. Smith <greg at krypto.org> wrote:

> It seems like the discussion so far is:
>>
>> Victor: "I know people when people hear 'new API' they get scared and
>> think we're going to do a Python-3-like breaking transition, but don't
>> worry, we're never going to do that."
>> Nathaniel: "But then what does the new API add?"
>> Greg: "It lets us do a Python-3-like breaking transition!"
>>
>
> That is not what I am proposing but it seems too easy for people to
> misunderstand it as such. Sorry.
>
> Between everything discussed across this thread I believe we have enough
> information to suggest that we can avoid an "everyone's afraid of a new 3"
> mistake by instead making a shim available with a proposed new API that
> works on top of existing Python VM(s) so that if we decide to drop the old
> API being public in the future, we could do so *without a breaking
> transition*.
>

I know that has always been my hope, especially if any new API is actually
going to be more restrictive instead of broader.


>
> Given that, I suggest not worrying about defining a new C API within the
> CPython project and release itself (yet).
>

+1 from me. Until we have a PEP outlining the actual proposed API I'm not
ready to have it go into 'master'. Helping show the shape of the API by
wrapping pre-existing APIs I think that's going to be the way to sell it.


>
> Without an available benefit, little will use it (and given the function
> call overhead we want to isolate some concepts, we know it will perform
> worse on today's VMs).
>
> That "top-5" module using it idea?  Maintain forks (hooray for git) of
> whatever your definition of "top-5" projects is that use the new API
> instead of the CPython API.  If you attempt this on things like NumPy, you
> may be shocked at the states (plural on purpose) of their extension module
> code.  That holds true for a lot of popular modules.
>
> Part of the point of this work is to demonstrate that non-incremental
> order of magnitude performance change can be had on a Python VM that only
> supports such an API can be done in its own fork of CPython, PyPy,
> VictorBikeshedPy, FbIsAfraidToReleaseANewGcVmPy, etc. implementation to
> help argue for figuring out a viable not-breaking-the-world transition plan
> to do such a C API change thing in CPython itself.
>

I think part of the challenge here (and I believe it has been brought up
elsewhere) is no one knows what kind of API is necessary for some faster VM
other than PyPy. To me, the only C API that would could potentially start
working toward and promoting **today** is one which is stripped to its bare
bones and worst mirrors Python syntax. For instance, I have seen
PyTuple_GET_ITEM() brought up a couple of times. But that's not syntax in
Python, so I wouldn't feel comfortable including that in a simplified API.
You really only need attribute access and object calling to make object
indexing work, although for simplicity I can see wanting to provide an
indexing API.

But otherwise I think we are making assumptions here. For me, unless we are
trying to trim the C API down to just what is syntactically supported in
Python and in such a way that it hides all C-level details I feel like we
are guessing at what's best for other VMs, both today and in the future,
until they can tell us that e.g. tuple indexing is actually not a problem
performance-wise.

And Just to be clear, I totally support coming up with a totally
stripped-down C API as I have outlined above as that shouldn't be
controversial for any VM that wants to have a C-level API.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20181116/02918191/attachment.html>