[Python-Dev] New Python Initialization API

[Steve Dower]

On 29Mar.2019 1830, Victor Stinner wrote:
> The purpose of the PEP 587 is to have a working document so everyone
> can look at the proposed API (stay focused to the API rather than
> bothering with the implementation). IMHO it's now time to get more
> people looking at the Python Initialization.
> 
>> But there are enough of us
>> with fuzzy but valid ideas in our heads that we really need that
>> brainstorming session to mix them together and find something feasible.
>> Maybe we're best to put it off until PyCon at this point.
> 
> Python 3.8 feature freeze is scheduled at the end of May, less than
> one month after the PyCon. It might be a little bit too late, no?

I don't think we want to rush this in for 3.8 at this point anyway. The
design of how Python is embedded is one of those things that could
drastically affect the scenarios it gets used for in the future
(probably half of my tasks at work right now involve embedding CPython),
so I'd like to get it right.

> Would you mind to elaborate these ideas?

I'd love to, but I don't have them all straight right now, and one of
the problems with putting them in writing is I don't get immediate
feedback when I'm not being clear enough or if there is missing context.
I know you personally have seen most of my ideas, because I keep pinging
you on them ;)

My big one is what I posted on capi-sig about being able to classify our
APIs better and define scenarios where they are ready for use, as well
as breaking up unnecessary dependencies so that embedders have more
flexibility (the rings and layers post). I posted a few examples of how
initialization "could" be on various bugs I've had to deal with relating
to it, and obviously I've been pushing the embeddable distro for Windows
for a while (which is surprisingly popular with a very specific subset
of users), as well as using it myself, so there are things that just
annoy me enough about what we currently have.

But I really do think this should start as a high bandwidth, in-person
brainstorm session to get through the first few big scenarios. Then
it'll be easy to open those up to review and let anyone submit their
needs for hosting Python. And once we've collated a good set of "needs"
we'll have a chance of designing the configuration and initialization
APIs that will satisfy most/all of them. Maybe in time for 3.9 (or 3.10,
if our RM gets the accelerated release cycle he wants ;) ).

I personally think being able to embed Python easily and safely in other
applications will be a powerful feature that will allow many
non-developers to write code to get their work done, as we already see
with Jupyter (and family). More are coming, but the responsibility is on
us to make it successful. I want to get it right.

Cheers,
Steve