Reinvent no more forever

[Ben Finney]

Howdy all,

On dirtSimple.org[0], PJE wrote:

    "Why is Python "blessed" with so much reinvention? Because it's
    often cheaper to rewrite than to reuse. Python code is easy to
    write, but hard to depend on. You pretty much have to:

        1. limit yourself to platforms with a suitable packaging
        system,
        2. bundle all your dependencies into your distribution, or
        3. make your users do all the work themselves

    Ouch. No wonder rewriting looks easier."

He then goes on to talk about his own solution: PyPI, setuptools,
EasyInstall, and so on for better dependency control. He concludes:

    "But for right now, I plan to start by putting my dependencies
    where my mouth is. I will bundle no more forever, for I am tired
    of keeping them up-to-date with their upstream maintainers. I
    shall rend my monolithic packages into smaller projects, so that
    people can make use of cool parts of PEAK without having to
    install anything but the part they want. I will find out if this
    stuff really works. From where the setuptools now stand, I will
    bundle no more forever."

Fine sentiments. What does this mean for PyPI though? How should those
of us who also want to "reinvent no more forever" proceed? How do we
deal with the rampant proliferation of a zillion implementations of
some standard idiom in PyPI?

The case that led me to this quandary is simple: I'm developing an
application that could become simple, and I'm on the lookout for
pieces that might be valuable outside this application. The first one
to catch my eye is an Enum implementation.

I now have a setuptools package of my Enum implementation. Should I
submit it to PyPI?

Advantages:

  - I can declare a dependency on that package for all my other
    packages that need such functionality, instead of bundling it
    every time.

  - Others can benefit from my code, instead of yet again including an
    Enum implementation (home-grown, or picked from a cookbook) by a
    simple dependency declaration.

  - The code hopefully gets improved and generalised and all the other
    benefits that accrue to software with many users.

Disadvantages:

  - Proliferation. What's the protocol when[1] someone else puts an
    (incompatible, differently-specified) Enum implementation into
    PyPI?

  - Naming. How many ways can a package providing an Enum be named?
    I'd prefer mine to be named "Enum" or "enum", but why should mine
    be the one that claims that name?

  - It's just a pretty simple type, with unit tests. Does this really
    justify a PyPI package?

This is common to any repository of code. But the "automatic
dependency" problem means that all those packages, and many more
outside that repository, need to know how those problems are resolved.

Operating system distributors have policies (to greater or lesser
degrees of strictness) to ensure this kind of quality control. My
understanding of PyPI is that there's no such policy.

I'd love to follow the mantra PJE espouses, but if it's good for one
person it's probably good for countless others. How do we deal with
that? What actions can we take in advance to prevent problems in
future?


[0] <URL:http://dirtsimple.org/2005/07/reinvent-no-more-forever.html>

[1] Of course, someone already has. I prefer mine to theirs, hence the
question.

-- 
 \         "I planted some bird seed. A bird came up. Now I don't know |
  `\                               what to feed it."  -- Steven Wright |
_o__)                                                                  |
Ben Finney