[Numpy-discussion] linux wheels coming soon

[Jonathan Helmus]

On 4/14/16 3:11 PM, Matthew Brett wrote:
> On Thu, Apr 14, 2016 at 12:57 PM, Matthew Brett <matthew.brett at gmail.com> wrote:
>> On Thu, Apr 14, 2016 at 12:25 PM, Jonathan Helmus <jjhelmus at gmail.com> wrote:
>>>
>>> On 4/14/16 1:26 PM, Matthew Brett wrote:
>>>> Hi,
>>>>
>>>> On Thu, Apr 14, 2016 at 11:11 AM, Benjamin Root <ben.v.root at gmail.com>
>>>> wrote:
>>>>> Are we going to have to have documentation somewhere making it clear that
>>>>> the numpy wheel shouldn't be used in a conda environment? Not that I
>>>>> would
>>>>> expect this issue to come up all that often, but I could imagine a
>>>>> scenario
>>>>> where a non-scientist is simply using a base conda distribution because
>>>>> that
>>>>> is what IT put on their system. Then they do "pip install ipython" that
>>>>> indirectly brings in numpy (through the matplotlib dependency), and end
>>>>> up
>>>>> with an incompatible numpy because they would have been linked against
>>>>> different pythons?
>>>>>
>>>>> Or is this not an issue?
>>>> I'm afraid I don't know conda at all, but I'm guessing that pip will
>>>> not install numpy when it is installed via conda.
>>> Correct, pip will not (or at least should not, and did not in my tests)
>>> install numpy over top of an existing conda installed numpy. Unfortunately
>>> from my testing, conda will install a conda version of numpy over top of a
>>> pip installed version.  This may be the expected behavior as conda maintains
>>> its own list of installed packages.
>>>> So the potential difference is that, pre-wheel, if numpy was not
>>>> installed in your conda environment, then pip would build numpy from
>>>> source, whereas now you'll get a binary install.
>>>>
>>>> I _think_ that Python's binary API specification
>>>> (pip.pep425tags.get_abi_tag()) should prevent pip from installing an
>>>> incompatible wheel.  Are there any conda experts out there who can
>>>> give more detail, or more convincing assurance?
>>> I tested "pip install numpy" in conda environments (conda's equivalent to
>>> virtualenvs) which did not have numpy installed previously for Python 2.7,
>>> 3.4 and 3.5 in a Ubuntu 14.04 Docker container.  In all cases numpy was
>>> installed from the whl file and appeared to be functional.  Running the
>>> numpy test suite found three failing tests for Python 2.7 and 3.5 and 21
>>> errors in Python 3.4. The 2.7 and 3.5 failures do not look concerning but
>>> the 3.4 errors are a bit strange.
>>> Logs are in
>>> https://gist.github.com/jjhelmus/a433a66d56fb0e39b8ebde248ad3fe36
>> Thanks for testing.  For:
>>
>> docker run -ti --rm ubuntu:14.04 /bin/bash
>>
>> apt-get update && apt-get install -y curl
>> curl -LO https://bootstrap.pypa.io/get-pip.py
>> python3 get-pip.py
>> pip install numpy nose
>> python3 -c "import numpy; numpy.test()"
>>
>> I get:
>>
>> FAILED (KNOWNFAIL=7, SKIP=17, errors=21)
>>
>> This is stock Python 3.4 - so not a conda issue.  It is definitely a
>> problem with the wheel because a compiled numpy wheel on the same
>> docker image:
>>
>> apt-get update && apt-get install -y curl python3-dev
>> curl -LO https://bootstrap.pypa.io/get-pip.py
>> python3 get-pip.py
>> pip install --no-binary=:all: numpy nose
>> python3 -c "import numpy; numpy.test()"
>>
>> gives no test errors.
>>
>> It looks like we have some more work to do...
> Actually, I can solve these errors by first doing:
>
> apt-get install gcc
>
> I think these must be bugs in the numpy tests where numpy is assuming
> a functional compiler.
>
> Does the conda numpy give test errors when there is no compiler?
>
> Cheers,
>
> Matthew

Yes, both the wheel and conda numpy packages give errors when there is 
not a compiler.  These errors clear when gcc is installed.  Looks like 
the wheels are fine, just forgot about a compiler.

Cheers,

     - Jonathan Helmus