[Distutils] [Python Language Summit] Distutils / Packaging survey

[Floris Bruynooghe]

On Fri, Jan 30, 2009 at 02:36:43PM -0500, P.J. Eby wrote:
> At 06:39 PM 1/30/2009 +0000, Floris Bruynooghe wrote:
>> I imagine things like libdir, prefix, datadir, docdir and other things
>> copied from autoconf.  Where the defaults would be something like:
>>
>> prefix = sys.prefix
>> libdir = sys.prefix/lib/pythonX.Y/site-packages/pkgname
>> datadir = sys.prefix/share/mypackage
>> docdir = sys.prefix/share/doc/mypackage
>
> I'm confused by the above lines: do you mean the *project* name, or the 
> name of some package within the project?  What if the project contains no 
> packages, only modules?  What is libdir for?

Oops, sorry.  That's even inconsistent in a few lines!

libdir = sys.prefix/lib/pythonX.Y/site-packages
datadir = sys.prefix/share/<project_name>
docdir = sys.prefix/share/doc/<project_name>

libdir's purpose is no more then saying where the modules and packages
go.  I can't think of any reason (right now) why a module/application
would want to know this, but the installation tool needs to know this.
http://wiki.python.org/moin/distutils-autoconf tries to split this up
into arch-independent and arch-dependent files (.py/.pyc/.pyo and
.so/.pyd) but I'm not sure if that will work since that creates the
namespaces problem of having an extension module inside a package with
normal modules.  It might be better to keep treating pure python
modules as arch-dependent, as is done now.


>> > from my current understanding, and having in mind to introduce such a
>> > file in a way things
>> > can gently evolve:
>> >
>> > What if an egg-info-like file was present in the package from the very
>> > beginning,
>> > describing the package metadata, and used by setup.py, *and* at build time
>> > by third party tools. (with setup.py knowing that it can be changed 
>> at run time)
>>
>> I don't see why moving all the metadata to egg-info would improve
>> things.
>
> Moving from code to data means better tool interoperability.  setup.py 
> sucks as a format for obtaining data, especially since many distutils 
> newbs hardcode all sorts of rubbish in their setup.py files, like writing 
> to files without paying attention to the command line.

Yes that's true, I agree.  I forgot about this argument and was just
thinking of what would be the smallest step for setup.py writers
(developers).

>>   You could easily(?) deprecate the `package_data' and
>> `data_files' keywords to setup() and replace them with `doc_files',
>> `data_files', `man_files', `config_files', etc.  And depending on
>> which --sysconfdir, --datadir, etc options where used to setup.py
>> install_egg_info would write the correct values for datadir etc.  The
>> runtime would then use your pkginfo.get_metadata() to find the files.
>
> ...in which case, why not just put all the files in the .egg-info to  
> begin with?

You're right, it may need more changes to distutils and especially
since both the old and new way will need to be supported at the same
time, but it will be better in the end.

> Meanwhile, getting rid of package_data or *requiring* a runtime API to 
> access files is going to be a major barrier to adoption in the short
> run.

It should not be *required*.  If someone keeps using the
`package_data' keyword it should just end up in $libdir like it used
too (and you will be able to find it using os.path and __file__), thus
not forcing anyone to change (until a packager gets fed up and submits
a patch ;-)).  It's just if someone *wants* to provide the extra
metadata and use an API to find the data at runtime then it's there in
a nice and sensible way.


Regards
Floris


-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org