[Numpy-discussion] Default builds of OpenBLAS development branch are now fork safe

[Julian Taylor]

FYI, binaries linking openblas should add this patch in some way:
https://github.com/numpy/numpy/pull/4580

Cliffs: linking OpenBLAS prevents parallelization via threading or 
multiprocessing.

just wasted a bunch of time figuring that out ... (though its well 
documented in numerous stackoverflow questions, too bad none of them 
reached us)

On 04/01/2014 02:59 AM, Matthew Brett wrote:
> Hi,
>
> On Wed, Mar 26, 2014 at 11:34 AM, Julian Taylor
> <jtaylor.debian at googlemail.com> wrote:
>> On 26.03.2014 16:27, Olivier Grisel wrote:
>>> Hi Carl,
>>>
>>> I installed Python 2.7.6 64 bits on a windows server instance from
>>> rackspace cloud and then ran get-pip.py and then could successfully
>>> install the numpy and scipy wheel packages from your google drive
>>> folder. I tested dot products and scipy.linalg.svd and they work as
>>> expected.
>>>
>>
>>>
>>> Would it make sense to embed the blas and lapack header files as part
>>> of this numpy wheel and make numpy.distutils.system_info return the
>>> lib and include folder pointing to the embedded libopenblas.dll and
>>> header files so has to make third party libraries directly buildable
>>> against those?
>>>
>>
>> as for using openblas by default in binary builds, no.
>> pthread openblas build is now fork safe which is great but it is still
>> not reliable enough for a default.
>> E.g. the current latest release 0.2.8 still has one crash bug on
>> dgemv[1], and wrong results zherk/zer2[2] and dgemv/cgemv[3].
>> git head has the former four fixed bug still has wrong results for cgemv.
>
> I noticed the Carl was only getting three test failures on scipy - are
> these related?
>
> ======================================================================
> FAIL: test_decomp.test_eigh('general ', 6, 'F', True, False, False, (2, 4))
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>    File "D:\devel\py27\lib\site-packages\nose\case.py", line 197, in runTest
>      self.test(*self.arg)
>    File "D:\devel\py27\lib\site-packages\scipy\linalg\tests\test_decomp.py",
> line 642, in eigenhproblem_general
>      assert_array_almost_equal(diag2_, ones(diag2_.shape[0]), DIGITS[dtype])
>    File "D:\devel\py27\lib\site-packages\numpy\testing\utils.py", line
> 811, in assert_array_almost_equal
>      header=('Arrays are not almost equal to %d decimals' % decimal))
>    File "D:\devel\py27\lib\site-packages\numpy\testing\utils.py", line
> 644, in assert_array_compare
>      raise AssertionError(msg)
> AssertionError:
> Arrays are not almost equal to 4 decimals
>
> (mismatch 100.0%)
>   x: array([ 0.,  0.,  0.], dtype=float32)
>   y: array([ 1.,  1.,  1.])
>
> ======================================================================
> FAIL: Tests for the minimize wrapper.
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>    File "D:\devel\py27\lib\site-packages\nose\case.py", line 197, in runTest
>      self.test(*self.arg)
>    File "D:\devel\py27\lib\site-packages\scipy\optimize\tests\test_optimize.py",
> line 435, in test_minimize
>      self.test_powell(True)
>    File "D:\devel\py27\lib\site-packages\scipy\optimize\tests\test_optimize.py",
> line 209, in test_powell
>      atol=1e-14, rtol=1e-7)
>    File "D:\devel\py27\lib\site-packages\numpy\testing\utils.py", line
> 1181, in assert_allclose
>      verbose=verbose, header=header)
>    File "D:\devel\py27\lib\site-packages\numpy\testing\utils.py", line
> 644, in assert_array_compare
>      raise AssertionError(msg)
> AssertionError:
> Not equal to tolerance rtol=1e-07, atol=1e-14
>
> (mismatch 100.0%)
>   x: array([[ 0.75077639, -0.44156936,  0.47100962],
>         [ 0.75077639, -0.44156936,  0.48052496],
>         [ 1.50155279, -0.88313872,  0.95153458],...
>   y: array([[ 0.72949016, -0.44156936,  0.47100962],
>         [ 0.72949016, -0.44156936,  0.48052496],
>         [ 1.45898031, -0.88313872,  0.95153458],...
>
> ======================================================================
> FAIL: Powell (direction set) optimization routine
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>    File "D:\devel\py27\lib\site-packages\nose\case.py", line 197, in runTest
>      self.test(*self.arg)
>    File "D:\devel\py27\lib\site-packages\scipy\optimize\tests\test_optimize.py",
> line 209, in test_powell
>      atol=1e-14, rtol=1e-7)
>    File "D:\devel\py27\lib\site-packages\numpy\testing\utils.py", line
> 1181, in assert_allclose
>      verbose=verbose, header=header)
>    File "D:\devel\py27\lib\site-packages\numpy\testing\utils.py", line
> 644, in assert_array_compare
>      raise AssertionError(msg)
> AssertionError:
> Not equal to tolerance rtol=1e-07, atol=1e-14
>
> (mismatch 100.0%)
>   x: array([[ 0.75077639, -0.44156936,  0.47100962],
>         [ 0.75077639, -0.44156936,  0.48052496],
>         [ 1.50155279, -0.88313872,  0.95153458],...
>   y: array([[ 0.72949016, -0.44156936,  0.47100962],
>         [ 0.72949016, -0.44156936,  0.48052496],
>         [ 1.45898031, -0.88313872,  0.95153458],...
>
> ----------------------------------------------------------------------
> Ran 8940 tests in 143.892s
>
>> Openblas is great if you do not have the patience to build ATLAS and
>> only use a restricted set of functionality and platforms you can easily
>> test.
>
> I don't think it's possible to build ATLAS on Windows 64-bit at the
> moment, and it would take a lot of work to make it build, and Clint W
> has said he does not want to invest much time maintaining the Windows
> build, so unless something changes, I think ATLAS is not a viable
> option - for 64 bits at least.
>