[Python-Dev] API for the new sysconfig module

[Nick Coghlan]

On Sat, Dec 11, 2010 at 7:59 AM, R. David Murray <rdmurray at bitdance.com> wrote:
> On Thu, 09 Dec 2010 16:18:14 -0800, Raymond Hettinger <raymond.hettinger at gmail.com> wrote:
>> ISTM it mostly could have been reduced to single call returning a nested dictionary.
>
> If what was returned was, as you suggested on IRC, a set of named tuples,
> it seems to me that would be a very convenient interface.  And since
> all values (as I understand it) are intended to be static (even if some
> aren't at the moment), it seems to make sense from an implementation
> standpoint as well.
>
> Like Éric, I'm not sure what the implications of the existing module
> having been released in 2.7 and 3.2 beta are in terms of making such an
> API change.

We've reverted from beta to pseudo-alpha after discovering
sufficiently egregious mistakes in the past. (we didn't *actually*
revert the build naming because release IDs that don't increase
monotonically can confuse matters). However, I don't think this is
sufficiently egregious to qualify.

My own reasons for advocating retention of the separate module:

1. Something equivalent to the current sysconfig module is going to be
needed to actually work out the contents of the data structure.
Rewriting all that in C would be silly, so the most that could happen
is to rename "sysconfig" to "_sysconfig".

2. Invoking Python code from sys module functions is possible but
untidy (generally involving implicit module imports). Since we need a
module anyway for implementation purposes, why not make it public?

3. The sysconfig module docs provide a nicer space to talk about the
purpose of the provided information rather than squirreling it away in
a corner of the sys module docs

4. "python -m sysconfig" provides a useful dump of the system status

5. The API design did not get much *new* thought, as the idea was to
allow users to do anything that the old distutils.sysconfig could do,
but without requiring them to depend on distutils to get that
functionality. The easiest way to ensure that is to just copy the
existing API rather than trying to get creative. Don't think of it as
adding a "whole new module", thinking of it as decoupling a relatively
small existing module (i.e. distutils.sysconfig) from a larger package
that isn't installed by default on all systems (i.e. distutils).

A concrete problem with Raymond's idea in particular is that there is
actually a full set of paths defined for *each scheme*. get_path() and
get_paths() let you ignore this most of the time, since there is an
appropriate default scheme defined for each platform, so they just use
that if you don't specify one explicitly. A named tuple could
obviously expose the paths for all of the schemes, with the paths for
the default scheme duplicated at the top level, but the data structure
starts looking very messy at that point. The named tuple idea also
conflates meta-data about the sysconfig data (the list of scheme names
and path names) with the actual data for the current platform
(platform, version, paths, config_vars).

So I think reusing the tried and tested API is easily justified in
this case. What may be desirable in 3.3 (particularly with the
distutils2 reintegration taking place), is to revisit the sysconfig
API to see if there are better ways to expose some of the data (such
as a function returning a named tuple containing the raw data
underlying the output of "python -m sysconfig").

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia