[Distutils] Unexpected VersionConflict

[PJ Eby]

On Fri, Aug 9, 2013 at 9:04 AM, Townsend, Scott E. (GRC-RTM0)[Vantage
Partners, LLC] <scott.e.townsend at nasa.gov> wrote:
> That does indeed fix this problem, but requiring an egg writer to
> interrogate all dependent packages (and their dependent packagesŠ) and
> then hoist the dependencies up won't be robust if those dependent packages
> change their requirements between the time the egg is written and the time
> it's loaded.

That's why it's generally left up to the application
installer/integrator to address these sorts of conflicts, and why it's
usually a bad idea for anybody to be requiring exact versions.  (I'd
suggest asking your dependency to not specify exact point releases,
too.)

There is one other possibility, though: have you tried reversing the
list of your project's dependencies so that the more-specific
project's dependencies are processed first?  (i.e., so that 1.5.2 will
be selected as "best" before the non-version-specific one is used)

That might fix it without requiring you to pin a version yourself.


> It seems to me that if a requirement has no version specified, then it
> shouldn't have a way to cause a VersionConflict. One possible way of
> implementing this would be to have resolve() only check that a
> distribution exists if no version is specified, do not update 'best'.
> 'to_activate' would need to be updated with 'generic' distributions only
> if a requirement with a version specifier hadn't been seen.

Thing is, the complete lack of a version requirement is pretty rare,
AFAIK, and so is the exact version match that's causing your problem.
The combination existing on the same library is therefore that much
rarer, so such a change would just be something of a complex kludge
that wouldn't improve any other use cases.

Probably a better way would be to change the version resolution
algorithm to be less "greedy", and simply rule out unacceptable
versions as the process goes along, then picking the most recent
versions left when everything necessary has been eliminated.
(Ideally, such an algorithm would still track which distributions had
the conflicting requirements, though.)

That would be a pretty significant change, but potentially worth
someone investigating.  There are some big downsides, however:

* It's not really a suitable algorithm for installation tools that
don't have access to a universal dependency graph, because they can't
tell what the next level of dependencies will be

* Recursion causes a combinatorial explosion, because what if you
select a different version and it has different dependencies
(recursively)?  Now you need backtracking, and there's a possibility
that the algorithm will take a ridiculous amount of time to still
conclude that there's nothing you can do about the conflict.

These drawbacks are basically why I just wrote a simple greedy match
algorithm in the first place, figuring it could always be improved
later if it turned out to be needed in practice.  There have been
occasional comments over the last decade or so by people with ideas
for better algorithms, but no actual code yet, as far as I know.