[Python-Dev] Making PyInterpreterState an opaque type

[Steve Dower]

On 16Feb.2019 1332, Antoine Pitrou wrote:
> On Sat, 16 Feb 2019 11:15:44 +0100
> Jeroen Demeyer <J.Demeyer at UGent.be> wrote:
>> On 2019-02-16 00:37, Eric Snow wrote:
>>> One thing that would help simplify changes
>>> in this area is if PyInterpreterState were defined in
>>> Include/internal.  
>>
>> How would that help anything? I don't like the idea (in general, I'm not 
>> talking about PyInterpreterState specifically) that external modules 
>> should be second-class citizens compared to modules inside CPython.
>>
>> If you want to break the undocumented API, just break it. I don't mind. 
>> But I don't see why it's required to move the include to 
>> Include/internal for that.
> 
> This sounds like a reasonable design principle: decree the API
> non-stable and prone to breakage (it already is, anyway), don't hide it.

As I was chatting with Eric shortly before he posted this, I assume the
idea would be to expose anything useful/necessary via a function. That
at least removes the struct layout from the ABI, without removing
functionality.

> It's true that in the PyInterpreterState case specifically, there
> doesn't seem much worthy of use by third-party libraries.

Which seems to suggest that the answer to "which members are important
to expose?" is "probably none". And that's possibly why Eric didn't
mention it in his email :)

This is mostly about being able to assign blame when things break, so
I'm totally okay with extension modules that want to play with internals
declaring Py_BUILD_CORE to get access to them (though I suspect that
won't work out of the box - maybe we should have a
Py_I_TOO_LIKE_TO_LIVE_DANGEROUSLY?).

I like that we're taking (small) steps to reduce the size of our API. It
helps balance out the growth and leaves us with a chance of one day
being able to have an extension model that isn't as tied to C's ABI.

Cheers,
Steve