[Python-Dev] Making PyInterpreterState an opaque type

[Antoine Pitrou]

On Mon, 18 Feb 2019 19:04:31 -0800
Steve Dower <steve.dower at python.org> wrote:
> 
> > I'm afraid of hiding actually useful private macros under Py_BUILD_CORE.
> > For example, Modules/_functoolsmodule.c and Modules/_json.c use API
> > functions from (4). But if an API function is useful for implementing
> > functools or json, then it's probably also useful for external extension
> > modules: what if I want to implement something similar to functools or
> > json, why shouldn't I be allowed to use those same API functions?
> > 
> > For a very concrete example, was it really necessary to put
> > _PyTuple_ITEMS in (4)? That's used in _functoolsmodule.c. Especially
> > given that the very similar PySequence_Fast_ITEMS is in (2), that seems
> > like a strange and arbitrary limiting choice.  
> 
> The reason to do this is that we can "guarantee" that we've fixed all
> users when we change the internal representation. Otherwise, the
> internal memory layout becomes part of the public ABI, which is what we
> want to fix. (PyTuple_GET_ITEM is just as problematic, FWIW.)

But PyTuple_GET_ITEM and PyList_GET_ITEM are important for performance,
as are other performance-oriented macros.

> If you always rebuild your extension for every micro version (3.x.y) of
> CPython, then sure, go ahead and use this.

Usually we would guarantee that API details don't change in bugfix
versions (i.e. the 3.x.y -> 3.x.(y + 1) transition).  Has that changed?
That may turn out a big problem for several third-party extensions...

Regards

Antoine.