[Distutils] Expectations on how pip needs to change for Python 3.4

[Donald Stufft]

On Jul 14, 2013, at 1:58 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> Either the existing APIs are moved to a different name, or they get declared stable and pip switches to "internally forked" APIs any time a backwards incompatible change is needed for refactoring purposes (see runpy._run_module_as_main for an example of needing to do this in the standard library). I've had to directly deal with too many issues arising from getting this wrong in the past for me to endorse bundling of a module that doesn't follow this practice with CPython - if introspection indicates an API is public, then it's public and subject to all standard library backwards compatibility guarantees, or else we take the pain *once* and explicitly mark it private by adding a leading underscore rather than leaving it in limbo (contextlib._GeneratorContextManager is a standard library example of the latter approach - it used to lack the leading underscore, suggesting it was a public API when it's really just an implementation detail of contextlib.contextmanager).
> 
> Mere documentation of public vs private generally doesn't cut it, as too many people use dir(), help() and inspect() rather than the published docs to explore APIs. The only general exception I'm aware of is "test" packages, including the standard library's test package, and for those you can make the case that having "test" or "tests" as a name segment is just as clear an indicator of something being private as at least one name segment starting with a leading underscore.
> 
> I really this is a fairly big ask for the pip maintainers, but I *don't* consider "Oh, don't use our module API, it isn't stable" to be an adequate answer for something that is bundled with the standard installers. Beyond that, I don't mind if the answer is to declare the 1.5 API stable or to sprinkle underscore where appropriate or moving everything to a private package - the documentation and the naming conventions just need to be consistent in their private vs public distinctions (although your points do suggest heavily that the right answer is to accept the burden of backwards compatibility for all APIs currently marked public, and move towards the introduction of appropriate private APIs over time through refactoring).

I agree with Noah here. In my eyes pip is either an external project with it's own polices on backwards compatibility and governance or it's part of the standard library and under the domain of Python core. I'm completely against moving it into the standard library for all the reasons I've given in the past.

Maybe I'm reading too much into this but one of my primary fears here is that including pip with the Python distribution is going to lead to Python core dictating to pip what pip must do with itself. Now I don't mean to say that Python core should have no sway over pip either as they are officially blessing it and including it as part of the official releases. If I am reading too much into it then I apologize.

I just want to make sure that the boundaries between the governance of Python and pip are clearly defined and the expectations on both sides are laid out and agreed upon before it happens. And I think this raises a good point about how the two projects are going to interact.

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20130714/d0b86e01/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 841 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20130714/d0b86e01/attachment.pgp>