[Python-Dev] Move ensurepip blobs to external place

[Paul Moore]

On 24 March 2018 at 10:50, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On 24 March 2018 at 18:50, Serhiy Storchaka <storchaka at gmail.com> wrote:
>>
>> Currently the repository contains bundled pip and setuptools (2 MB total)
>> which are updated with every release of pip and setuptools. This increases
>> the size of the repository by around 2 MB several times per year. There were
>> total 37 updates of Lib/ensurepip/_bundled, therefore the repository
>> contains up to 70 MB of unused blobs. The size of the repository is 350 MB.
>> Currently blobs takes up to 20% of the size of the repository, but this
>> percent will likely grow in future, because they where added only 4 years
>> ago.
>>
>> Wouldn't be better to put them into a separate repository like Tcl/Tk and
>> other external binaries for Windows, and download only the recent version?
>
>
> Specifically, I believe that would entail adding them to
> https://github.com/python/cpython-bin-deps, and then updating the make file
> to do a shallow clone of the relevant branch and copy the binaries to a
> point where ensurepip expects to find them?
>
> I'm fine with the general idea of moving these out to the bin-deps repo, as
> long as cloning the main CPython repo and running "./configure && make &&
> ./python -m test test_ensurepip" still works. We'd also want to add docs to
> the developer guide on how to update them (those docs are missing at the
> moment, since the update process is just dropping the new wheel files
> directly into the right place)

I don't have a problem with moving the pip/setuptools wheels - as long
as (as a pip dev doing a release) I know where to put the files, it
makes little difference to me. But as Nick says, if the files aren't
in the main CPython repository, the build process (both the Unix and
the Windows processes) will need updating to ensure that the files are
taken from where they do reside and put in the right places.

I'd assume that a change like that is big enough that it would be
targeted at 3.8, BTW (and so won't affect what I need to do for 3.7).

Paul