[Python-ideas] More "ensure*" packages

[Donald Stufft]

On August 13, 2015 at 1:08:11 PM, Steve Dower (steve.dower at python.org) wrote:
> I'd like to propose expanding the list of 3rd-party packages we bundle
> and install by default. (Obviously this does not apply to platforms that
> repackage Python and can do whatever they want, but on Windows and Mac
> we are fully responsible for these.)
>  
> Currently, we bundle pip (and some of its dependencies - let's avoid
> that particular discussion right now please, it's on python-dev) and
> install it by default in a way that lets users easily update to the
> latest version. Including pip in the standard library would lock users
> into a specific version for the lifetime of that Python version, which
> would be a bad thing.
>  
> From my point-of-view, this has been very successful in Python 2.7, 3.4
> and will also be successful in 3.5. For Python 3.6, I'd like to do a
> similar thing with:
>  
> * requests
> * tkinter (including tcl/tk, IDLE, and other dependencies)
>  
> Given the language summit discussion at PyCon this year, I think
> requests is easy to justify. (Quick summary for those who weren't there:
> we'd love to include requests in the stdlib, but it's too important and
> needs much more frequent updates.) Preinstalling a given version in a
> way that allows updates (and maybe attempting an update on installation)
> sounds great to me.
>  
> tkinter is worth more discussion :) For the remainder of this email,
> I'll use "tkinter" as shorthand to refer to Tcl, Tk, Tix, _tkinter,
> tkinter, idlelib/IDLE, PyDoc, turtledemo and any other dependencies or
> dependents that I missed.
>  
> In my experience, few Python scripts depend on or assume tkinter is
> available. tkinter is already an optional item in the Windows installer
> (maybe Mac too? I don't know) and there are certainly installations of
> Python out there that don't have it. From this side, nothing would
> actually change by installing tkinter into site-packages rather than Lib.
>  
> (One impact may be the start menu shortcuts for IDLE and PyDoc, but
> provided the entry points into those tools are kept stable we can
> continue adding shortcuts from the installer. People who omit tkinter
> and then install it later would not get shortcuts. But since they
> omitted it from the installer, they probably don't want them - they
> likely just got a package that has tkinter as a dependency.)
>  
> IDLE is already allowed to make enhancements in maintenance branches
> (https://www.python.org/dev/peps/pep-0434/), and we have recently
> received patches that are to be applied to *four* branches. The freedom
> to enhance IDLE is greatly improved by making it a PyPI installable
> package and disconnecting it from the stdlib's schedule.
>  
> How this would actually be structured is up for discussion. I believe
> the change can be made without sacrificing anything, and the resulting
> flexibility will be worth it.
>  
> Thoughts?
> 

One possible thing to look at for prior art, is what Haskell does. They don’t have a bunch of ensure* modules or anything like it, instead they have their compiler (which is like “Haskell Core” and then on top of that they layer a bunch of libraries (Called “Haskell Platform”). This platform releases every ~6 months and just includes something like 40 different libraries with it that represent common development tools and widely used libraries [1].

So I guess my question is, instead of continuing down a path where we add more ensure* style modules to the standard library, why not do something similar and have “Python the Language” and “The Python Platform”, and the platform would be the Python language + N “important” or “popular” packages. This could release on a quicker release schedule than Python itself (since it would really be more like a meta package than anything that itself got developed) and would give the ability to ship things like this without the problems that we’ve had with ensurepip. From a downstream perspective they would just package all of this stuff as normal and it would just be available as normal. We could even publish a metapackage on PyPI that had no code of it’s own, but existed simply to list all of the platform packages as dependencies (with ==) and then people could easily depend on the Python “platform” in their own code.

This would essentially involve someone(s) needing to be the gatekeeper of which libraries become part of the Python platform, some small packaging shims to handle the metapackage on PyPI, and then the installer stuff for OSX and Windows (probably nothing for other OSs? Or maybe a tarball? I don’t know).


[1] https://www.haskell.org/platform/contents.html

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA