[Python-Dev] Packaging and binary distributions for Python 3.3

[Paul Moore]

On 16 October 2011 22:32, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> There's one area of pysetup3 functionality which I don't think has been
> discussed in this thread, though it's pertinent to Windows users. Namely, a
> completely declarative approach to installation locations will not satisfy all
> requirements. For example, if you want to install PowerShell scripts, the right
> way of doing that is to make Shell API calls at installation time to determine
> the correct location for the installing user.

Interesting. That's not a use case that I have encountered, but I can
see it could be an issue. I have been working on the basis that a
bdist_simple format that matches the functionality of bdist_wininst is
at least good enough for those projects that currently use
bdist_wininst. The only potential issue right now is with postinstall
scripts, which bdist_simple doesn't support. It would be easy enough
to add, and indeed it may be possible to use existing packaging
functionality already (I haven't looked into this area).

> I have this working at present for
> a project, using pysetup's hook functionality; but any move to a completely
> passive archive format would lose this kind of flexibility. So, I think whatever
> archive format we end up with should provide exactly the same level of
> flexibility we currently get with pysetup3 for pure-Python projects, but
> extended to include binary deliverables.

I don't disagree, but I'm struggling to see how that would be done.

> The simplest way of doing this is to
> register those binaries in setup.cfg, and to have hook code check for the
> correct dependencies in the run-time environment before actually installing
> (e.g. x86/x64/ARM/Python version dependencies). While it's not the slickest
> solution imaginable, it does allow for just about all use cases.

Can you give an example of a setup.cfg, that would work like this?
Suppose I have two files, foo.py and bar.pyd, which are a pure-python
module and a compiled C extension. How would I write a setup.cfg and
lay out the directory structure to allow pysetup install to do the
right thing? I tried to do this myself, but couldn't get it to work
the way I expected. It may be I was just hitting bugs, but it felt to
me like I was going at the problem the wrong way.

> This
> flexibility is more important under Windows than under Posix, because the
> installation locations under Posix conform much more closely to a standard (FHS)
> than anything you find in the Windows environment.

Agreed in principle, although in practice, most projects I've
encountered have ended up working within bdist_wininst limitations,
probably just as a matter of practicality.

> Distribution build step = get all source, binary code and data defined in
> setup.cfg and in a single directory tree (containing the setup.cfg in the root
> of the tree), then zip that tree.
>
> Distribution installation step = unzip aforementioned zip, and run pysetup3
> install <unzip_location>.

Works for me. I'd like to see a bdist_xxx command to do the build step
as you describe, if only to make it trivially simple for developers to
produce binary distributions. Having to package stuff up manually is
bound to put at least some developers off. If you can give me the
example I mentioned above, I could work on modifying the bdist_simple
code I posted to the tracker today to produce that format rather than
my custom format based on bdist_wininst.

For the installation step, you shouldn't even need to unzip, as
pysetup3 can do the unpacking for you.

> BTW, I believe that for virtual env installations, there's no need to provide
> integration with Add/Remove Programs. I find that the Add/Remove Programs dialog
> takes long enough to populate as it is :-(

Agreed. Personally, as I've said, I'm happy not to use Add/Remove even
for system installations - pysetup list and pysetup remove do what I
need without slowing down the Add/Remove list. But I accept that's not
likely to be the view of many Windows users. Anyone using virtual
envs, though, is probably by definition comfortable enough with
command line tools to be willing to use pysetup3.

Paul.