[SciPy-dev] Re: [SciPy-user] Install failure of SciPy

[Pearu Peterson]

Hi Travis,

On Fri, 18 Oct 2002, Travis Oliphant wrote:

> >
> > I hope that after all the trouble, you'll not give up on SciPy, especially
> > when I have another bad news about the Intel compiler support.
> >
> > First, to fix the above problem, insert the following line
> >
> > #define isnan cephes_isnan
> >
> > to files
> >
> >   special/cephes/mconf_LE.h
> >   special/cephes/mconf_BE.h
> >
> 
> Should this be changed in the source tree.  It looks like other
> platforms/compilers have an isnan already defined.

Yes. At least under linux, isnan is a macro (what about Windows, Sun
platforms?), so, I renamed cephes isnan() to cephes_isnan() and using it
only when macro isnan is not defined.

> > and re-run the build command that should finish now successfully.
> >
> > The bad news is that when running scipy tests, it will segfault after
> >
> > .Testing ncf
> >
> 
> This is definitely a special function problem.  This is the "first-place"
> in the scipy testing that one calls out to a (hand-wrapped) fortran-compiled special function.
> I'm sure the problem is with different linking needs, here.

I'm not sure that the problem is with linking. Only few functions actually
give segfaults. Other seem to work fine.

> The special function module needs a c-function to call so that's why f2py
> was not used.

You mean that wrapper functions execute additional C commands after
returning from C/Fortran function? In principle, f2py can handle this (I
have used it in few places in linalg wrappers) but I'd like to introduce
better hooks for this in f2py before others start to use it...

>  I'm not sure how to get setup to use the right compiler
> switches so that the FORTRAN code called in cdf_wrappers.c works on many
> platforms.

I don't understand the problem. As I said above, it seems that only few
functions cause the trouble with ifc (I haven't tested them all, though).

> > in the stats module (I have tested it both on suse and debian with
> > Intel 5.0 compiler). All other tests from other modules seem to pass fine.
> >
> > I am not sure yet what function, even what module exactly, is causing this
> > segfault, but it must be either stats or special (as stats uses special).
> 
> I'm sure it's special.

I agree.

> Try calling special.ncfdtr all by itself and seeing what happens.

Here follows some debugging info:

>>> import libwadpy
WAD Enabled
>>> import cephes
>>> cephes.ncfdtr(1,1,1,1)
WAD: Collecting debugging information...
WAD: Segmentation fault.
#0   0x403cfb98 in select_types()

In GDB:
>>> import cephes
>>> cephes.ncfdtr(1,1,1,1)
(no debugging symbols found)...
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 1024 (LWP 3214)]
0x4038ab98 in select_types (self=0x0, 
    arg_types=0x3ff00000 <Address 0x3ff00000 out of bounds>,
data=0x80f3560, 
    function=0x0) at Src/ufuncobject.c:235
235                 for(j=0; j<self->nin; j++) {


> > And both modules cannot be tested locally (special even doesn't have
> > working unittests)
> 
> Unittests for special are being written as we speak (I've got an undergrad
> working on it).

Great! I have introduced few simple testing hooks already. I hope they
will not get into a way.

> > as they import scipy and that makes finding/fixing bugs
> > very time consuming and difficult. I also tried to remove the dependencies
> > of these modules on scipy
> 
> I don't think stats should ever have dependency on scipy removed (it will
> always need special for example).
> 
> Special should be able to depend only on scipy_base.

Note that special.orthogonal uses linalg.eig, otherwise it seems to be
rather independent.

OT:
Hmm, I have long time had in mind implementing lazy import
hooks so that modules will be imported only when some of its function is
acctually going to be used. This would reduce scipy import time
considerably (sometimes it takes 5-15 seconds to just import scipy) and
would make scipy more accesible for simple scripts that use only few scipy
features.
The general idea would be the following:

m = lazy_import('m')  # lazy equivalent of 'import m'
# m would be a lazy_import class (maybe subclass of module class) instance
# that will be stored also in sys.modules, it contains only python state
# (sys.path, etc.) needed to import m.

m.foo()
# When accessing m attribute for the first time, the following would
# happen:
# 1) m is removed from sys.modules
# 2) 'import m' is executed
# 3) local lazy_import instance `m' is replaced with module `m' instance


> >
> > So, unless someone will figure out quickly the source of this segfault,
> > the Intel compiler remains unsupported. But I'll keep looking for the
> > solution ...
> >
> 
> Again, I think it has to do with magic C-to-Fortran calling conventions
> for the Intel compiler.
> 
> Pearu,
> 
> You will recognize the
> 
> #if defined(NO_APPEND_FORTRAN)
> #if defined(UPPERCASE_FORTRAN)
> #define F_FUNC(f,F) F
> #else
> #define F_FUNC(f,F) f
> #endif
> #else
> #if defined(UPPERCASE_FORTRAN)
> #define F_FUNC(f,F) F##_
> #else
> #define F_FUNC(f,F) f##_
> #endif
> #endif
> 
> code in the C-wrappers (remember these are not Python wrappers) for the
> fortran code in cdflib and specfun.

I don't understand the relevance of your notes. AFAIK, Intel compiler does
not use magic for C-to-Fortran calling more than any other compiler.

Pearu