[Distutils] How to get pip to really, really, I mean it -- rebuild this damn package!

[Chris Barker]

Context:

I'm maintaining a number of conda packages of various packages, some  of
which are mine, some others, some pure python, some extensions, etc.

The way conda build works is you  specify some meta data, and a build
script(s), and conda:

sets up an isolated environment in which to build.
installs the build dependencies
runs teh build script
see's what got installed, and makes a package of it.
(there are complications, but that's the idea)


so what to do in the build script for a python package? the simple anser is:

$PYTHON setup.py install

But then you get those god- awful eggs, or if it's not a setuptools built
package, you don't get the right meta data for pip, etc. to resolve
dependencies.

[NOTE: I do want all the pip compatible meta data, otherwise, you have pip
trying to re-instll stuff, etc if someone does install something with pip,
or pip in editable mode, or...]

so some of us have started doing:

$PYTHON setup.py install --single-version-externally-managed  --record
record.txt

Which mostly seems to work -- though that is a God-awful command line to
remember....

And it fails if the package has a plain old distuitls-based setup.py

so I started going with:

$PYTHON -m pip install ./

and that seemed to work for awhile for me. However, I've been having
problems lately with pip not bulding and re-installing the package. This is
really weird, as the conda build environment is a clean environment, there
really isn't a package already installed.

here is the log:

+ /Users/chris.barker/miniconda2/envs/_build/bin/python -m pip install -v ./

Processing /Users/chris.barker/miniconda2/conda-bld/work/gsw-3.0.3

  Running setup.py (path:/tmp/pip-umxsOD-build/setup.py) egg_info for
package from file:///Users/chris.barker/miniconda2/conda-bld/work/gsw-3.0.3

    Running command python setup.py egg_info

  Source in /tmp/pip-umxsOD-build has version 3.0.3, which satisfies
requirement gsw==3.0.3 from
file:///Users/chris.barker/miniconda2/conda-bld/work/gsw-3.0.3

  Requirement already satisfied (use --upgrade to upgrade): gsw==3.0.3 from
file:///Users/chris.barker/miniconda2/conda-bld/work/gsw-3.0.3 in
/Users/chris.barker/miniconda2/conda-bld/work/gsw-3.0.3

Requirement already satisfied (use --upgrade to upgrade): numpy in
/Users/chris.barker/miniconda2/envs/_build/lib/python2.7/site-packages
(from gsw==3.0.3)

Requirement already satisfied (use --upgrade to upgrade): nose in
/Users/chris.barker/miniconda2/envs/_build/lib/python2.7/site-packages
(from gsw==3.0.3)

Building wheels for collected packages: gsw

  Running setup.py bdist_wheel for gsw ...   Destination directory:
/tmp/tmprPhOYkpip-wheel-

  Running command /Users/chris.barker/miniconda2/envs/_build/bin/python -u
-c "import setuptools,
tokenize;__file__='/tmp/pip-umxsOD-build/setup.py';exec(compile(getattr(tokenize,
'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))"
bdist_wheel -d /tmp/tmprPhOYkpip-wheel- --python-tag cp27

done

  Stored in directory:
/Users/chris.barker/Library/Caches/pip/wheels/51/4e/d7/b4cfa75866df9da00f4e4f8a9c5c35cfacfa9e92c4885ec5c4

  Removing source in /tmp/pip-umxsOD-build

Successfully built gsw

Cleaning up...

You are using pip version 8.0.1, however version 8.0.2 is available.

You should consider upgrading via the 'pip install --upgrade pip' command.

So it seems to think it's already installed -- huh? what? IN any case, it
doesn't install anything. It looks like it's referencing some cache, or
manifest or something outside of the python environment itself. So if I
installed it in a different Anaconda environment, it gets confused here.

(BTW, I can replicate this behavior outside of conda build by creating a
new conda environment by hand, and trying to ues pip to build a package
locally)

So I tried various command-line options:

$PYTHON -m pip install -I -v --upgrade --no-deps ./
but no dice.

I also tried --no-cache-dir -- no change.

So how can I tell pip that I really do want it to bulid and install this
dran package from source, damn it!

Other option -- go back to:

$PYTHON setup.py install --single-version-externally-managed  --record
record.txt
And have to fight with pip only for the non-setuptools packages. Does the
--single-version-externally-managedcommand do ayting different than pip?

Thanks,

-Chris











-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160128/3d02dd30/attachment.html>