[Distutils] Re: PEP 262

[Phillip J. Eby]

At 09:04 AM 3/8/04 -0500, A.M. Kuchling wrote:
>On Sun, Mar 07, 2004 at 10:26:38PM -0500, Phillip J. Eby wrote:
> > Hi Andrew.  I was looking over PEP 262 today for possible 
> implementation in
> > "setuptools", and I had a few questions and comments.
>
>You should have sent this post to the catalog-sig or distutils-sig, because
>it raises some interesting issues.
>
> > 1) be called 'install-db', to avoid conflict with a package name
>
>Good idea; PEP updated.
>
> > 2) be potentially present on every directory in sys.path, with the 
> contents
> > merged in sys.path order.  This would allow home-directory or other
> > alternate-location installs to work, and ease the process of a distutils
> > install command writing the file.
>
>Hm; for some reason I find the idea of multiple databases disturbing; would
>it be too confusing and error-prone?  On the other hand, it does mean that
>package manager tools can take into account packages that a user has
>privately installed, which is a nice feature.  On balance I suppose it's a
>good idea.
>
>Ooh, but what does setup.py do if it's told to install packages to a
>directory not on sys.path?  Does it write an install-db directory to the
>directory it's told to write to, or does it do nothing?

The directory where it's installing packages to.  Also, for purposes of 
uninstallation, it should include that installation directory on the list 
of directories scanned for package installation info.  This all means that 
the installdb API needs to be able to be told what directories to search, 
perhaps with a default of sys.path.

In a manner analogous to the way modules can shadow each other on sys.path, 
distributions found earlier in the directory searches should shadow later 
distributions with the same name.  (This prevents confusion between a 
user's local version of a distribution, and the system-supplied version.)


> > Next, I'd like to ask if it's okay to have the minimum contents of a
> > package file be PKG-INFO FILES instead of also requiring PROVIDES and
> > REQUIRES.  The latter two seem a bit premature for
>
>PROVIDES and REQUIRES can simply be left empty, though.  I can weaken the
>language in this section a bit.
>
> > * Should the package-database file itself be included in the files
> > list?  (I would think yes, but of course it can't contain its own 
> checksum!)
>
>Does it really matter?  The only case I can think where this matters is
>removing a package, where you could just read all of the files listed in the
>database and delete them.  But the code implementing this will be in the
>InstallationDatabase class, which will already have the filename of the
>database file.  In short, I don't think this really matters.  Do you have a
>use case for including it?

If you use 'setup.py install --result=somefile', the distribution's 
information file should be included (for compatibility with external 
packaging tools that use --result).  So, that means the 'install' command 
object should have it listed in its 'get_outputs()'.  But, presumably the 
list that goes into the file is going to *come* from 'get_outputs()'.  So, 
it seems like the list would include the file itself, if that makes sense.



> > * The spec says that checksums of .pyc/.pyo files should not have their
> > checksums stored, but does not say *how* to not store them.  By omitting
> > the field?  By placing 'unknown' there?
>
>I was thinking that the field would simply be '-'.  It could be omitted, but
>then if we ever add another column there will be problems.  PEP updated to
>specify "-".

Come to think of it, why *shouldn't* those files be verified as to 
checksum?  Is it simply because they contain an internal timestamp?


>Hey, this raises another issue; should we guarantee the format of
>installation databases remains compatible across Python versions, or is it
>subject to arbitrary change?

I would think it needs to be guaranteed.  If there's a change, it would 
need to be by adding a new section name like 'FILES2' for the new 
format.  In practice, though, FILES is extensible via additional columns, 
and the metadata section is extensible via adding new header lines.  So, 
all in all, the format is reasonably future-proof, at least for the 
critical sections.


> > * Are there any legal or other issues regarding deployment of SHA1?  I'm
>
>No, SHA just does hashing, not encryption, so there are no problems with its
>inclusion in Python.
>
> > Finally, there's a bit of a terminology issue.  I think that the term
> > "package" should be used for Python packages exclusively, using perhaps
> > "distribution" to refer to an installed unit.  So, instead of looking up a
> > "package", you would look up a "distribution".  Currently, the use of
> > "package" in the PEP is ambiguous, making it difficult to understand in
> > parts.
>
>PEP updated.
>
> > Anyway, apart from these issues/questions, I think it should be possible
> > for me to move forward with an implementation, possibly complete with a
> > facility to uninstall/upgrade/verify, although I can't guarantee *when*
>
>Note that there's already an implementation in nondist/sandbox/pep262 in
>CVS, though I don't remember how complete it was.  I'm planning to dust it
>off before PyCon.

Ah.  If I'd known where it was, I'd have UTSL'ed to see if it answered any 
of these questions.  I'll take a look at it when I get a chance.

Another spec question: is there a blank line after the line listing the 
sections?  Can there be one?  Are sections separated by a *single* blank 
line, or can there be more than one blank line between them?