[Distutils] Clarifying the meaning of the sysconfig "data" directory

[Nick Coghlan]

On 8 December 2013 08:22, Paul Moore <p.f.moore at gmail.com> wrote:
> On 7 December 2013 21:32, Marcus Smith <qwcode at gmail.com> wrote:
>> actually, bdist_wheel doesn't handle absolute paths in the "data_files"
>> keyword like standard setuptools installs do, which honor it as absolute
>> (which seems to match the examples in the distutils docs)
>> when using absolute paths, the data ends up in the packaged wheel at the top
>> level (not in the {distribution}-{version}.data folder).
>> And get's installed relative to site-packages (along with the project
>> packages)
>>
>> so, bdist_wheel is re-interpreting distutil's "data_files" differently.
>> maybe better for wheel to fail to build projects with absolute data_files,
>> than to just reinterpret it like that.
>
> +1. It seems to me (from a Windows user's perspective) that projects
> that use absolute paths (or relative paths that go above the
> installation root) are fundamentally different from ones that don't.
> For a start they feel more like "applications" than "Python packages".

This started with my suggestion to create a separate directory in in
the wheel format for bundling arbitrary other files:

    https://bitbucket.org/pypa/pypi-metadata-formats/issue/13/add-a-new-subdirectory-to-allow-wheels-to

Daniel initially suggested that such a thing would be redundant with
the existing "data" directory in sysconfig, but given the subsequent
discussions, I no longer think that is true.

A new scheme directory would be a simpler solution than changing the
wheel format, though.

> Maybe what we want is a sysconfig path "root" that explicitly encodes
> the filesystem root.

s/filesystem root/installation root/

I'm assuming that's what you meant, since you'd know better than I do
that "filesystem root" isn't a particularly useful concept on Windows
:)

I guess "drive root" would be another way to interpret it, but that
doesn't seem useful either.

> Then we can require that *all* paths are relative
> to some sysconfig path, and must not use ".." to go "up" from there.
> Projects that use the "root" path can be clearly noted as not
> wheel-compatible (and quite likely not cross-platform).

Well, they'd be wheel compatible - the wheels would just need to be
marked as platform specific.

> By making it an explicit path, we can make the policy explicit and
> provide better error messages and metadata.

Agreed, it sounds like "data" is currently being used to mean two
different things, and we need to define a new scheme directory to
separate out the FHS compliance use case.

> Paul
>
> PS Please note that as a Windows user, I don't have any practical
> experience of projects with requirements for absolue paths - they are
> alien to the Windows world. So my views should be taken in that light
> (i.e, as being of limited value :-)).

Yeah, absolute paths are mostly about playing nice with FHS, which is
mostly about playing nice with standardised system administration
tools. For a server:

- /var/* is the mutable data you need to back up
- /etc/* is the config to be controlled by your config management system
- most everything else should be controlled by the system package manager

Distro packagers and system administrators get grumpy when upstream
software doesn't play by those rules, because it makes things harder
to audit, harder to back up, and just generally harder to administer.

Perhaps rather than trying to be generic with a name like "root", we
could just define an explicit "fhs" path?

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia