[Python-Dev] Keyword meanings [was: Accept just PEP-0426]

[PJ Eby]

On Sun, Dec 9, 2012 at 12:54 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Sun, Dec 9, 2012 at 6:18 AM, PJ Eby <pje at telecommunity.com> wrote:
>>
>> On Sat, Dec 8, 2012 at 5:06 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> > On Sat, Dec 8, 2012 at 4:46 PM, PJ Eby <pje at telecommunity.com> wrote:
>> >>
>> >> So if package A includes a "Conflicts: B" declaration, I recommend the
>> >> following:
>> >>
>> >> * An attempt to install A with B already present refuses to install A
>> >> without a warning and confirmation
>> >> * An attempt to install B informs the user of the conflict, and
>> >> optionally offers to uninstall A
>> >>
>> >> In this way, any collateral damage to B is avoided, while still making
>> >> the intended "lack of support" declaration clear.
>> >>
>> >> How does that sound?
>> >
>> >
>> > No, that's not the way it works. A conflict is always symmetric, no
>> > matter
>> > who declares it.
>>
>> But that *precisely contradicts* what you said in your previous email:
>>
>> > It's to allow a project to say
>> > *they don't support* installing in parallel with another package.
>>
>> Just because A doesn't support being installed next to B, doesn't mean
>> B doesn't support being installed next to A.  B might work just fine
>> with A installed, and even be explicitly supported by the author of B.
>>  Why should the author of A get to decide what happens to B?  Just
>> because I trust A about A, doesn't mean I should have to trust them
>> about B.
>
>
> If I'm installing both A *and* B, I want to know if *either* project doesn't
> support that configuration. The order in which they get installed should
> *not* have any impact on my finding out that I am using one of my
> dependencies in an unsupported way that may cause me unanticipated problems
> further down the line.

This is probably moot now, but I didn't propose that installation
order matter -- in both scenarios I described, you end up with a
warning and A not installed, regardless of whether A or B were
installed first.

> The author of A *doesn't* get to decide what happens to B, *I* do.

The reason I said, "(de facto)", is because the default behavior of
whatever the next big installation tool is, would be what most users
would've gotten by default.

> They're
> merely providing a heads up that they believe there are problems when using
> their project in conjunction with B. My options will be:
> - use them both anyway (e.g. perhaps after doing some research, I may find
> out the conflict relates solely to a feature of B that I'm not using, so I
> simply update my project documentation to say "do not use feature X from
> project B, as it conflicts with dependency A")
> - choose to continue using A, find another solution for B
> - choose to continue using B, find another solution for A
>
> As a concrete example, there are projects out there that are known not to
> work with gevent's socket monkeypatching, but people don't know that until
> they try it and it blows up in their face.

Here's the question, though: who's going to maintain that list?

I can see gevent wanting to have a compatibility chart page in their
docs, but it seems unlikely they'd want to block installation of
non-gevent-compatible projects or vice versa.  Similarly, I can't see
why any of those other projects would want to block installation of
gevent, or vice versa.

That being said, I don't object to having the ability for either of
them to do so: the utility of the field is *much* enhanced once its
connection to installation tools is gone, since a wider variety of
issues can be described without inconveniencing users.


> I now agree that *enforcing* a conflicts field at install time in a Python
> installer doesn't make any sense, since the nature of Python means it will
> often be easy to sidestep any such issues once you're aware of their
> existence (e.g. by avoiding gevent's monkeypatching features and using
> threads to interact with the uncooperative synchronous library, or by
> splitting your application into multiple processes, some using gevent and
> others synchronous sockets). I also believe that *any* Conflicts declaration
> *should* be backed up with an explicit explanation and rationale for that
> conflict declaration in the project documentation.

Beyond that, I think a reference URL should be included *in the field
itself*, e.g. to a bug report, support ticket, or other page that
documents the incompatibility and will be updated as the situation
changes.  The actual usefulness of the field to anyone "downstream"
seems greatly reduced if they have to go hunting for the information
explaining the compatibility issue(s).

This is a good example of what I meant about clear thinking on
concrete use cases, vs. simply copying fields from distro tools.  In
the distro world, these kinds of fields reflect the *results* of
research and decision-making about compatibility.  Whereas, in our
"upstream" world, the purpose of the fields is to provide downstream
repackagers and integrators with the source materials for such
research.


> So, I still like the idea of including a Conflicts field, but think a few
> points should be made clear:
> - the Conflicts field would be for documenting other distributions which
> have known issues working together in the same process and thus constitute
> an unsupported configuration
> - this field would be aimed at package *users*, rather than at installation
> tools (although it would still be good if they installation tools supported
> scanning a set of packages for known conflicts)
> - any use of this field should be backed up with a more detailed explanation
> in the project documentation

My concrete recommendation based on your comments, then, is:

* The field should be called Known-Incompatibilities (to better
clarify its purpose and avoid confusion with similarly-named
installation-oriented metadata in other tools)

* The field should be of the form (though not necessarily syntax):

  ProjectName==incompatible_version; info=url

That is, each entry lists a project name and a specific version that
is known to be incompatible, along with a (required) information URL.
The URL should be for:

* a page that is updated with any change in the situation
* that will remain available indefinitely, and
* describes the specific reason that particular project is considered
incompatible, along with any available workarounds

For minor issues, a bug report or support ticket is acceptable;
otherwise, a long-lived documentation link should be used.  In-page
anchor links are acceptable.  A simple link to either project's home
page or main documentation page is *not* acceptable: the link must to
be a part of the documentation that directly addresses the nature of
the incompatibility.

I'm not too picky about the version specification approach, though;
the simplest thing is to only allow a single version to be named, but
it also seems it could be reasonable to list one or more version
ranges that appy, as long as they are not open-ended going forward.
That is, saying versions 1.1-2.3 are incompatible is ok, but not "1.1
on".  (Because the author of A is not in a position to declare on B's
behalf that the incompatibility will *never* be fixable.)

(I might be overthinking the versions, bit, though, since this is
really just about warnings.)

I would recommend that tools automatically provide the warning in
cases where a project C depends on versions of A and B that are
declared incompatible.  In this case, while one cannot *prove* the
incompatibility to be an issue, it is still a potential issue.  (This
is more of a package build-time issue, though, as with Replaced-By.)

Speaking of Replaced-By, it probably makes sense to require a URL in
the field there as well, but that URL can be an unchanging page such
as an archived post to a mailing list or blog, announcing the
project's renaming or obsolescence, and providing migration help or
links thereto.

I think it also should be a multi-valued field, just like
Known-Incompatibilities.  Recently, I came across a Python project
"lepl" (a parser combinator library) that just declared its
end-of-life, and actually recommended multiple alternatives, each of
which would be more appropriate for some uses of a parsing library.
(That is, there was no single "does everything" replacement for lepl's
full feature set.)

Finally, the PEP should document that the audience for both the
Replaced-By and Known-Incompatibilities fields is developers and
system integrators (such as distro teams).  So they are designed to be
processed by tools that *build* packages, rather than tools that
*install* them.

So, if you build a project that depends on something that's replaced,
or a pair of things known to be incompatible, that's when you get
warnings and such.  Tools to check such things on installed projects
are also ok, though to avoid unnecessary warnings, it's probably best
to only list incompatibilities for co-dependents (and orphaned
replaced projects) by default.

That is, a checker should probably ignore replacements when there's an
installed project depending on the replaced version, and ignore
incompatibilities that aren't part of the same requirements subtree
(and thus unlikely to be used together).  Of course, having options to
be more verbose is not an issue, and this isn't really something to
legislate anyway -- it's just that listing *every* replaced project or
potentially-incompatible pairing in even a moderately-sized
installation is likely to be far more noise than signal.