[Distutils] "Python Package Management Sucks"

[Phillip J. Eby]

At 09:40 PM 10/1/2008 +0200, Josselin Mouette wrote:
>Le mercredi 01 octobre 2008 à 14:39 -0400, Phillip J. Eby a écrit :
> > >   We need to be able to mark locale, config, and data files in
> > >the metadata.
> >
> > Sure...  and having a standard for specifying that kind of
> > application/system-level install stuff is great; it's just entirely
> > outside the scope of what eggs are for.
>
>I don’t follow you. If the library needs these files to work, you
>definitely want to ship them, whether it is as their FHS locations in a
>package, or in the egg.

Egg files aren't an all-purpose distribution format; they were 
designed for application plugins, and for libraries needed to support 
application plugins.  As such, they're self-contained and weren't 
designed for application-level installation support, such as 
documentation, configuration or data files, icons, etc.

As has been pointed out, these are deficiencies of .egg files wrt the 
full spectrum of library and application installation needs, which is 
why I'm pushing for us working on an installation metadata standard 
that can accommodate these other needs that the .egg layout isn't 
really suited for.

My main point about the resources is simply that it's a needless 
complication to physically separate static data needed by a library 
at runtime, based solely on its file extension, in cases where only 
that library will be reading that file, and the file's contents are 
constant for that version of the library.

To put it another way, if some interpretation of the FHS makes a 
distinction between two files encoding the same data, one named 
foo.bar and foo.py, where the only difference between the two is the 
internal encoding of the data, then that interpretation of the FHS is 
not based on any real requirement, AFAICT.

Of course, for documentation, application icons, and suchlike, the 
data *will* be read by things other than the library itself, and so a 
standardized location is appropriate.  The .egg format was designed 
primarily to support resources read only by the package in question, 
and secondarily to support metadata needed by applications or 
libraries that the package "plugs in" to.  It was not originally 
intended to be an general-purpose system package installation format.


> > To be clear, I mean here that a "file" (as opposed to a resource) is
> > something that the user is expected to be able to read or copy, or
> > modify.  (Whereas a resource is something that is entirely internal
> > to a library, and metadata is information *about* the library itself.)
>
>It’s not as simple as that. Python is not the only thing out there, and
>there are many times where your resources need to be shipped in existing
>formats, in files that land at specific places. For example icons go
>in /usr/share/icons, locale files in .mo format in /usr/share/locale,
>etc.

And docs need to go in /usr/share/doc, I presume.  But these aren't 
necessarily "resources" in the way I'm defining the term.  Some of 
them *could* be, perhaps.  Others aren't.

To be clear, what I'm trying to say is that it is a perfectly valid 
use case for a Python package author to have static data contained 
within their Python package directory layout for purposes of 
accessing that data as if it were code, but without having to go to 
the trouble of converting it to a .py file (and possibly having to 
extract it back out at runtime).  This usage of "data" files isn't in 
conflict with the FHS, as I understand it.

But I also understand that there are other kinds of "data" files 
which *don't* fall under that use case, and which it is desirable to 
install to shared locations.  We need to support both.