[Numpy-discussion] Compiling numpy using icc gets missing library error

[Hoyt Koepke]

> Thanks for the thorough description of getting everything to work.

You're welcome.  I'm glad people find it helpful :-).

>> -ipo -- okay for linker / c++, but not elsewhere.  For C, it causes
>> the long_double_representation() function in setup_common() to fail
>> (I'll note this on the corresponding ticket), and causes f2py to fail
>> to generate correct wrappers.
>
> Would it be enough to put a comment in the intelccompiler.py code not
> to use the -ipo flag, or do you think there's something to fix here?
> Doesn't seem to be a new problem by the way:
> http://cens.ioc.ee/pipermail/f2py-users/2006-January/001229.html

Well, to be honest, it's debatable how this should be fixed, if at
all.  The issue is that the intermediate object files and archives are
in their own format -- hence the need for xiar instead of ar -- and I
suspect that that's what's messing up f2py and the
long_double_representation() stuff (I don't actually know how either
really works, so I could be wrong here).  Reverse engineering those
formats doesn't make sense.  Now it's possible that there may be a
better way of determining long_double_representation(), but I don't
know.  IMO it's not worth it.

>> -O3 -- okay for C++ / fortran, but not C.  For C, it causes einsum to hang.
>
> -O3 is the default optimization level, so this is a bug I guess.
> There's also another report in #1378 that -O3 doesn't work with numpy
> 1.4.0 (which does not have einsum). Should it be lowered to -O2 by
> default?

I'm not sure what's going on here.  I think it is likely a bug in icc
brought out by the really low-level code in einsum.  However, the
documentation said that the extra optimizations enabled by -O3 are
likely not going to help except in large, predictable loops where more
aggressive loop transformations are doable and help.  Thus my vote is
to put in O3 for C++ and fortran, but O2 on C (because of this bug).
However, if the einsum bug gets fixed, nice! (BTW, I think it's also
be possible to change per-file optimization settings with intel
specific pragmas, but I don't know for sure.   If so, that could be a
workaround.)

>> Fortran: -static -DMKL_LP64 -mkl -xHOST -O3 -fPIC  -fp-model strict
>> C:     -static -DMKL_LP64 -mkl -xHOST -O2 -fno-alias -fPIC -fp-model strict
>> C++: -static -DMKL_LP64 -mkl -xHOST -O3 -fno-alias -fPIC -fp-model strict
>> Linker: -DMKL_LP64 -mkl -xHOST -O2 -fno-alias -fPIC -fp-model strict
>> -openmp -lpthread
>
> I'm not sure which of those flags would be appropriate as a default in
> distutils, perhaps only fp-model-strict? If you could help put
> together a patch for numpy.distutils, that would be very helpful I
> think. The rest of your description could be put at
> http://scipy.org/Installing_SciPy/Linux.

-fp-model strict and -fno-alias, as it's analogous to the
-fno-strict-aliasing gcc flag required by python.  The -static flag is
probably optional.  -xHOST is the same as gcc's --march=native, so it
seems like the nature of the distutils system isn't appropriate.

I'll try to get a patch together for the distutils.  However, I'd want
someone to review it since I'm not that confident in my knowledge of
the distutils code.  I can also try to turn this into a more complete
description for the wiki.

-- Hoyt

++++++++++++++++++++++++++++++++++++++++++++++++
+ Hoyt Koepke
+ University of Washington Department of Statistics
+ http://www.stat.washington.edu/~hoytak/
+ hoytak at gmail.com
++++++++++++++++++++++++++++++++++++++++++