[Python-Dev] Making PyInterpreterState an opaque type

[Victor Stinner]

Hi Eric,

IMHO the main blocker issue for any C API change is that nobody is
able to measure the risk these changes. To better control the risk, I
propose to select a list of popular C extensions, and build a CI to
run their test suite on top of the development version of Python. Such
CI wouldn't detect all backward incompatible changes. It wouldn't
prevent us to merge backward incompatible changes. Some projects are
already tested on the master branch of Python.

My intent is to detect issues earlier, and if something goes wrong:
discuss early to decide what to do. Fixing only some popular C
extensions is one option. Another option is to provide some commands /
hints to help maintainers of C extensions to adapt their code to the
new C API. The other obvious option is to revert the change and maybe
do it differently.

Right now, it's too scary to walk in the dark.

What I also would like to see is the creation of a group of people who
work on the C API to discuss each change and test these changes
properly.

Victor

Le sam. 16 févr. 2019 à 00:41, Eric Snow <ericsnowcurrently at gmail.com> a écrit :
>
> Hi all,
>
> I've been working on the runtime lately, particularly focused on my
> multi-core Python project.  One thing that would help simplify changes
> in this area is if PyInterpreterState were defined in
> Include/internal.  This would mean the type would be opaque unless
> Py_BUILD_CORE were defined.
>
> The docs [1] already say none of the struct's fields are public.
> Furthermore, Victor already moved it into Include/cpython (i.e. not in
> the stable ABI) in November.  Overall, the benefit of making internal
> types like this opaque is realized in our ability to change the
> internal details without further breaking C-API users.
>
> Realistically, there may be extension modules out there that use
> PyInterpreterState fields directly.  They would break.  I expect there
> to be few such modules and fixing them would not involve great effort.
> We'd have to add any missing accessor functions to the public C-API,
> which I see as a good thing.  I have an issue [2] open for the change
> and a PR open.  My PR already adds an entry to the porting section of
> the 3.8 What's New doc about dealing with PyInterpreterState.
>
> Anyway, I just wanted to see if there are any objections to making
> PyInterpreterState an opaque type outside of core use.
>
> -eric
>
> p.s. I'd like to do the same with PyThreadState, but that's a bit
> trickier [3] and not one of my immediate needs. :)
>
>
> [1] https://docs.python.org/3/c-api/init.html#c.PyInterpreterState
> [2] https://bugs.python.org/issue35886
> [3] https://bugs.python.org/issue35949
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/vstinner%40redhat.com



-- 
Night gathers, and now my watch begins. It shall not end until my death.