[Distutils] buildout.dumppickedversions and buildout_versions

[Jim Fulton]

On Thu, Jan 3, 2013 at 8:03 AM, Marius Gedminas <marius at pov.lt> wrote:
> On Thu, Jan 03, 2013 at 07:35:20AM -0500, Jim Fulton wrote:
>> On Sat, Dec 15, 2012 at 11:56 AM, Chris Withers <chris at simplistix.co.uk> wrote:
>> > Yeah, the stuff that buildout.dumppickedversions and buildout_versions
>> > provide should just be in the core.
>> >
>> > Jim, if I worked up a pull request would you merge it?
>>
>> Maybe :)  I tried to look at buildout.dumppickedversions and
>> buildout_versions and I couldn't quite tell what they were for or how
>> they were different.  Are they different?  Or are they 2 versions of
>> the same thing?
>
> buildout-versions is a clone of buildout.dumppickedversions with a
> rather better choice of name.  I don't remember Chris's rationale for
> rewriting it instead of contributing to buildout.dumppickedversions.
>
> The problem is this: people tend to release new package versions to
> PyPI, and they tend to sometimes break backwards compatibility
> (intentionally or unintentionally), and then your existing
> buildouts start to fail.
>
> Buildout has version pinning to combat this, but you have to manage
> version pins by hand.  It's easy to miss a dependency or two when you
> add a new package to your buildout, or upgrade to a newer version that
> introduces an indirect dependency.

If you run buildout in verbose mode (-v) then you get output that's easy to
grep, sort, uniq and cut into new versions, but I agree that something easier
would be better.

>
> Buildout-versions solves the problem of missing version pins in two
> ways: it can warn you when your buildout uses packages without a version
> pin, and it can automatically update your version pins with those
> packages.
>
> It's enough to add
>
>   [buildout]
>   extensions = buildout-versions
>
> to your buildout.cfg to get a warning if any packages that are installed
> by buildout do not have a version pin.  I imagine it could be a buildout
> core feature named, e.g., 'warn-about-picked-versions = true' (a
> non-fatal version of 'allow-picked-versions = false').
>
> If you also add
>
>   buildout_versions_file = versions.cfg
>
> it'll append new version pins to versions.cfg with the versions that
> are picked while you run bin/buildout.  It's not smart -- you have to
> ensure the file you're writing to ends with a [versions] section, and
> that your mail buildout.cfg specifies `extends = versions.cfg` and that
> you have `[buildout] versions = versions` in either of the config files.
> You can append directly to buildout.cfg if you're careful enough to keep
> the [versions] section at the bottom, but I prefer a separate
> versions.cfg file to reduce clutter and make it easier to grep.
>
> As far as I can tell, buildout.dumppickedversions always overwrites the
> versions.cfg file, while buildout-versions only appends new version
> pins.  The append-only behaviour makes it easier to add meaningful
> comments like "# zc.somepackage 1.3.7 fixes this important bug about X"
> above version pins when you update them manually.
>
> This could be a builtin feature called, e.g., 'append-picked-versions-to
> = versions.cfg'.

Well, it sounds like this could be improved, especially based on our
experience to
date.

I'm going to give this some thought and start another thread to discuss ideas.

Jim

-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton
Jerky is better than bacon! http://zo.pe/Kqm