[Distutils] [setuptools] When .py files are "data", strange things happen

[David Handy]

On Thu, Jan 05, 2006 at 12:07:39PM -0500, Phillip J. Eby wrote:
> At 10:54 PM 1/4/2006 -0500, David Handy wrote:
> >I want setuptools to install a bunch of sample scripts in a 
> >package-relative
> >directory. Each end user will later run a script that copies these to a
> >directory relative to his or her home directory.
> >
> >However, strange things happen to data files that have a .py extension.
> >
> >1) Saying "include_package_data = True" was not enough to get them included
> >   in the egg. The .txt files were included, but the .py files were
> >   magically excluded.
> >
> >2) I overcame the above problem with the following setup parameter:
> >
> >        package_data = {
> >            # Include all .py files in the data directories, they
> >            # are excluded by default.
> >            'cpif.data': ['lander/*.py', 'samples/*.py'],
> >            }
> >
> >   But now I get a bogus .pyc file generated for each of these sample
> >   scripts. These bogus .pyc files get included in the egg.
> 
> If I understand your use case correctly, I don't see how that hurts 
> anything.

My project is not the typical software component - it is a kit for beginning
programmers. They are following the directions in my book. The .py "data"
files I refer to are a whole bunch of simple sample programs that users are
supposed to refer to as they go through my book. Really, they're
documentation more than code.

Most of my users won't navigate to the package directory itself, but instead
will work with copies of those files in their "MyPrograms" directory. But
those who do poke around (like I always do), and know a little bit of
Python, would wonder, "why are these single file programs, that are never
imported, compiled to .pyc files anyway? Am I missing something?" I didn't
want the answer to be "they're compiled because the author didn't try to
learn how to use setuptools properly." :) But it's not a huge issue.

> 
> The simplest way to solve your problem, however, is to put the samples in a 
> subdirectory of your project's .egg-info directory, and then use the 
> metadata APIs to access them.  .py files under .egg-info do not get 
> compiled.
> 
> Meanwhile, it's not really practical to have .py files inside a package be 
> included, but not compiled.  So your three choices are:
> 
> * Give the files a different extension (e.g. .py.orig or .py.in or .cfg or 
> whatever), which will then trivially work with include_package_data=True

I prefer to keep the names intact to make the packaging itself as
transparent as possible.

> * Include them explicitly as .py files, as shown above (and then ignore the 
> irrelevant .pyc files)

That's the path of least resistance right now. The welcome script that
copies the files to the user's MyPrograms directory will ignore .pyc files.

> * Put them in a subdirectory of the project's .egg-info, and then use the 
> pkg_resources metadata APIs at runtime to get access.
> 

I'll probably do this eventually. Right now I am not installing setuptools
with my package, just using it to create the installer.

Thanks for the advice!

-- 
David Handy
Computer Programming is Fun!
Beginning Computer Programming with Python
http://www.handysoftware.com/cpif/