[Catalog-sig] PyPI's external packages

[M.-A. Lemburg]

John J Lee wrote:
> On Mon, 16 May 2011, M.-A. Lemburg wrote:
> [...]
>> Just to clarify: creating an sdist really only means copying
>> over the files from the MANIFEST into a temporary dir and then
>> running tar or zip on the temporary directory. You can tell
>> distutils to keep the temporary dir around by using the --keep-temp
>> option on sdist. This will reduce the operation to just running
>> the archiver on the temp dir again.
>>
>> The same can be done for bdist_* commands by having distutils
>> skip the build process.
>>
>> Since I don't see how such trivial operations can affect the
>> quality of the code, the discussion sounds a bit artificial :-)
> 
> Perhaps a file gets left behind the second time somehow that wasn't
> present the first time, or vice versa (and MANIFEST includes that file).
> If the presence or absence of that file breaks stuff, you end up
> uploading a broken file.
> 
> Doing your release in an automated way makes this less likely, but
> anybody who has maintained a complicated buildbot setup knows that this
> kind of thing does happen.
>
>> Note that if you want to do full testing of a release, you'll
>> have to download the package from PyPI and then run the tests
>> on the downloaded archive to be sure that things work - everything
>> else is just preparation for this final run :-)
> 
> The upload step is interesting one from the PoV of testing because
> performing it implies "this is the release, come and get it".  There are
> other such steps (tagging, sending announcements), but this is one of
> them.  So it's nice to be as confident as possible at this point that
> you didn't screw it up.

Right, but the only way to find out is by downloading
what you uploaded and testing that package :-)

There are lots of things that can go wrong. In my experience,
most of them are user errors, e.g. you forgot to update the
MANIFEST file, bump the version, remove debug settings, etc.

The chance of two consecutive runs of sdist creating different
archives is rather small, compared to those sources of error.

That said, it's easy to get the upload command to use an
already created distribution file for the upload: just
add a new distutils command which sets .distribution.dist_files
to what list of files you want to upload.

This could also be added as an option to the distutils
upload command, so that you can run:

python setup.py upload --dist-files=dist/my-release-1.2.3.*

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, May 16 2011)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2011-06-20: EuroPython 2011, Florence, Italy               35 days to go

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/