[Import-SIG] File based configuration for the import system

[Nick Coghlan]

On 27 April 2016 at 01:26, Guido van Rossum <guido at python.org> wrote:
> On Mon, Apr 25, 2016 at 7:19 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>>
>>
>> This isn't a fully thought out idea, but I figured it was worth posting
>> here as a potentially improved motivation for the interpreter bootstrapping
>> improvements proposed in PEP 432.
>>
>> Currently, the proposed motivation there is for a system Python executable
>> that has different defaults from the normal CPython runtime:
>> https://www.python.org/dev/peps/pep-0432/#a-system-python-executable
>>
>> It turns out that isn't a particularly compelling motivation for the
>> degree of change proposed, since Christian Heimes was able to implement the
>> -I switch for isolated mode without too much hassle, and you can fairly
>> easily provide a wrapper script that implicitly sets -I for all invocations.
>>
>> For me though, one of the other motivations behind the proposed changes in
>> PEP 432 is getting to replace the current sys.path calculation code in
>> getpath.c and getpathp.c with Python code in importlib._bootstrap or
>> (depending on boot sequence details) importlib._bootstrap_external.
>
> Isn't code that lives in importlib._bootstrap just about as hard to maintain
> as code written in C?

It's somewhere in the middle - for _bootstrap, you're limited to the
builtins and the injected os and sys modules, while
_bootstrap_external is free to import any builtin and frozen modules
it needs.

The main pain with getpath.c in its current form is that when it runs
we don't even have the builtin *types* available, so it's all written
in terms of wchar_t* and char*. Even being able to replace those with
PyUnicode and PyList would be a respectable step forward, but still
not quite as nice as stepping up to the Python level and also
benefiting from the implicit memory management.

>> Once we did that, there would be a range of ideas (like allowing
>> additional path handling customisations in venv configuration files) that
>> become significantly more feasible to implement and maintain.
>>
>> So, does that sound reasonable to folks as a revised primary motivation
>> for that PEP: moving the default sys.path calculation code to Python for
>> ease of reuse across implementations, and for ease of maintenance and
>> enhancement within CPython itself?
>
> Much of the sys.path calculation (everything related to packages) is already
> in site.py. And you can skip it with -S.

Aye, the C level initialisation only covers finding the standard
library and the initial setting of sys.path[0].

That's one of the big reasons this refactoring proposal has lingered
for so long - the status quo for bootstrapping the interpreter isn't
particularly elegant, but it *works*, and most of the things folks
want to do can already be achieved in other ways.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia