distutils, sdist and tests

[Robert Kern]

Steven Bethard wrote:
> Robert Kern wrote:
>> Steven Bethard wrote:
>>> How do I get distutils to include my testing module in just the "sdist" 
>>> distribution?
>> Use a MANIFEST.
>>
>>   http://docs.python.org/dist/source-dist.html

Also, I just noted this tidbit:

"""If you don't supply an explicit list of files (or instructions on how to
generate one), the sdist command puts a minimal default set into the source
distribution:
  ...
  * anything that looks like a test script: test/test*.py (currently, the
Distutils don't do anything with test scripts except include them in source
distributions, but in the future there will be a standard for testing Python
module distributions)
  ...
"""

So you can just stick test_argparse.py into a test/ directory. Having tested
this, it appears to work (see below).

>>> I want test_argparse.py to be available in the source distribution, but 
>>> I don't think it should be included in the binary distributions.
> 
> Using a MANIFEST appears to do the same thing as putting "test_argparse" 
> into py_modules -- that is, it puts "test_argparse.py" into both "sdist" 
> and "bdist" distributions. In "bdist" distributions it gets installed to 
> the site-packages directory like any other module.

Are you sure that you don't have changes left over in your setup.py when you
tested that? Also check that you clear out your build/ directory every time. I
think the bdist_* commands tend to just copy whatever is in the appropriate
build/lib.*/ directory even though you've changed how those files should be
generated.

This is what I get with setup.py unmodified. The only thing I added was
MANIFEST.in and an empty test_argparse.py file.

[argparse]$ ls
MANIFEST.in      PKG-INFO         README.txt       argparse.py      setup.py
     test_argparse.py
[argparse]$ cat MANIFEST.in
include PKG-INFO
include *.txt
include *.py
[argparse]$ python setup.py bdist_dumb
running bdist_dumb
running build
running build_py
creating build
creating build/lib
copying argparse.py -> build/lib
installing to build/bdist.macosx-10.3-fat/dumb
running install
running install_lib
creating build/bdist.macosx-10.3-fat
creating build/bdist.macosx-10.3-fat/dumb
creating build/bdist.macosx-10.3-fat/dumb/Library
creating build/bdist.macosx-10.3-fat/dumb/Library/Frameworks
creating build/bdist.macosx-10.3-fat/dumb/Library/Frameworks/Python.framework
creating
build/bdist.macosx-10.3-fat/dumb/Library/Frameworks/Python.framework/Versions
creating
build/bdist.macosx-10.3-fat/dumb/Library/Frameworks/Python.framework/Versions/2.5
creating
build/bdist.macosx-10.3-fat/dumb/Library/Frameworks/Python.framework/Versions/2.5/lib
creating
build/bdist.macosx-10.3-fat/dumb/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5
creating
build/bdist.macosx-10.3-fat/dumb/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages
copying build/lib/argparse.py ->
build/bdist.macosx-10.3-fat/dumb/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages
byte-compiling
build/bdist.macosx-10.3-fat/dumb/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/argparse.py
to argparse.pyc
running install_egg_info
Writing
build/bdist.macosx-10.3-fat/dumb/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/argparse-0.5.0-py2.5.egg-info
creating /Users/rkern/src/argparse/dist
tar -cf /Users/rkern/src/argparse/dist/argparse-0.5.0.macosx-10.3-fat.tar .
gzip -f9 /Users/rkern/src/argparse/dist/argparse-0.5.0.macosx-10.3-fat.tar
removing 'build/bdist.macosx-10.3-fat/dumb' (and everything under it)
[argparse]$ tar ztf dist/argparse-0.5.0.macosx-10.3-fat.tar.gz
./
./Library/
./Library/Frameworks/
./Library/Frameworks/Python.framework/
./Library/Frameworks/Python.framework/Versions/
./Library/Frameworks/Python.framework/Versions/2.5/
./Library/Frameworks/Python.framework/Versions/2.5/lib/
./Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/
./Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/
./Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/argparse-0.5.0-py2.5.egg-info
./Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/argparse.py
./Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/argparse.pyc
[argparse]$ python setup.py sdist
running sdist
reading manifest template 'MANIFEST.in'
writing manifest file 'MANIFEST'
creating argparse-0.5.0
making hard links in argparse-0.5.0...
hard linking PKG-INFO -> argparse-0.5.0
hard linking README.txt -> argparse-0.5.0
hard linking argparse.py -> argparse-0.5.0
hard linking setup.py -> argparse-0.5.0
hard linking test_argparse.py -> argparse-0.5.0
tar -cf dist/argparse-0.5.0.tar argparse-0.5.0
gzip -f9 dist/argparse-0.5.0.tar
removing 'argparse-0.5.0' (and everything under it)
[argparse]$ tar ztf dist/argparse-0.5.0.tar.gz
argparse-0.5.0/
argparse-0.5.0/argparse.py
argparse-0.5.0/PKG-INFO
argparse-0.5.0/README.txt
argparse-0.5.0/setup.py
argparse-0.5.0/test_argparse.py


And this is what I get when I put test_argparse.py into a test/ directory:

[argparse]$ ls
PKG-INFO    README.txt  argparse.py setup.py    test
[argparse]$ python setup.py bdist_dumb
running bdist_dumb
running build
running build_py
creating build
creating build/lib
copying argparse.py -> build/lib
installing to build/bdist.macosx-10.3-fat/dumb
running install
running install_lib
creating build/bdist.macosx-10.3-fat
creating build/bdist.macosx-10.3-fat/dumb
creating build/bdist.macosx-10.3-fat/dumb/Library
creating build/bdist.macosx-10.3-fat/dumb/Library/Frameworks
creating build/bdist.macosx-10.3-fat/dumb/Library/Frameworks/Python.framework
creating
build/bdist.macosx-10.3-fat/dumb/Library/Frameworks/Python.framework/Versions
creating
build/bdist.macosx-10.3-fat/dumb/Library/Frameworks/Python.framework/Versions/2.5
creating
build/bdist.macosx-10.3-fat/dumb/Library/Frameworks/Python.framework/Versions/2.5/lib
creating
build/bdist.macosx-10.3-fat/dumb/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5
creating
build/bdist.macosx-10.3-fat/dumb/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages
copying build/lib/argparse.py ->
build/bdist.macosx-10.3-fat/dumb/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages
byte-compiling
build/bdist.macosx-10.3-fat/dumb/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/argparse.py
to argparse.pyc
running install_egg_info
Writing
build/bdist.macosx-10.3-fat/dumb/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/argparse-0.5.0-py2.5.egg-info
creating /Users/rkern/src/argparse/dist
tar -cf /Users/rkern/src/argparse/dist/argparse-0.5.0.macosx-10.3-fat.tar .
gzip -f9 /Users/rkern/src/argparse/dist/argparse-0.5.0.macosx-10.3-fat.tar
removing 'build/bdist.macosx-10.3-fat/dumb' (and everything under it)
[argparse]$ tar ztf dist/argparse-0.5.0.macosx-10.3-fat.tar.gz
./
./Library/
./Library/Frameworks/
./Library/Frameworks/Python.framework/
./Library/Frameworks/Python.framework/Versions/
./Library/Frameworks/Python.framework/Versions/2.5/
./Library/Frameworks/Python.framework/Versions/2.5/lib/
./Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/
./Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/
./Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/argparse-0.5.0-py2.5.egg-info
./Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/argparse.py
./Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/argparse.pyc
[argparse]$ python setup.py sdist
running sdist
warning: sdist: manifest template 'MANIFEST.in' does not exist (using default
file list)
writing manifest file 'MANIFEST'
creating argparse-0.5.0
creating argparse-0.5.0/test
making hard links in argparse-0.5.0...
hard linking README.txt -> argparse-0.5.0
hard linking argparse.py -> argparse-0.5.0
hard linking setup.py -> argparse-0.5.0
hard linking test/test_argparse.py -> argparse-0.5.0/test
tar -cf dist/argparse-0.5.0.tar argparse-0.5.0
gzip -f9 dist/argparse-0.5.0.tar
removing 'argparse-0.5.0' (and everything under it)
[argparse]$ tar ztf dist/argparse-0.5.0.tar.gz
argparse-0.5.0/
argparse-0.5.0/argparse.py
argparse-0.5.0/PKG-INFO
argparse-0.5.0/README.txt
argparse-0.5.0/setup.py
argparse-0.5.0/test/
argparse-0.5.0/test/test_argparse.py

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco