[SciPy-Dev] Release blocker?

[Ralf Gommers]

On Sat, Feb 19, 2011 at 5:09 PM, Christoph Gohlke <cgohlke at uci.edu> wrote:
>
>
> On 2/19/2011 12:22 AM, Ralf Gommers wrote:
>> On Thu, Feb 17, 2011 at 8:39 AM, Robert Kern<robert.kern at gmail.com>  wrote:
>>> On Wed, Feb 16, 2011 at 18:25, Pauli Virtanen<pav at iki.fi>  wrote:
>>>> On Wed, 16 Feb 2011 17:53:45 -0600, Robert Kern wrote:
>>>
>>>>> That said, distutils should already be passing the right flags to mingw
>>>>> to link against msvcr90.dll unless if we are interfering. Can you show
>>>>> us the command lines that are being executed on your machine? I believe
>>>>> that using the environment variable DISTUTILS_DEBUG=1 will cause
>>>>> setup.py to print them out as it goes along.
>>>>
>>>> Build log is here: http://pav.iki.fi/tmp/build.log
>>>> It's a simple Numpy 1.5.1 + current mingw32 + python.org 3.1 setup.
>>>>
>>>> Apparently, what happens is that modules linked with g++ get -lmsvcr90,
>>>> whereas modules linked with gfortran (due to BLAS, apparently) don't get
>>>> it.
>>>
>>> Ah, that makes sense. numpy.distutils owns the link flags for Fortran
>>> modules. I think the problem is in fcompiler/gnu.py:get_libraries()
>>> (both of them). We don't add the right runtime library unless if the
>>> configured C compiler is 'msvc'. I think we need to add it
>>> unconditionally.
>>>
>> I tried that, https://github.com/rgommers/numpy/tree/mingw-runtime-1.5.x.
>> Then building scipy against this patched numpy 1.5.1 shows that also
>> g77 receives '-lmsvcr90'. Build log at http://pastebin.com/Zu4PB4ss.
>> The tests all pass for me under Wine. I don't have access to a Win7
>> system though, so can someone who does try the installer in
>> http://sourceforge.net/projects/scipy/files/scipy/temp/?
>
> Just tried it on Windows 7. Qhull.pyd fails to load with "ImportError:
> DLL load failed". Process Monitor reports it is failing to find/load a
> file named '<stdin>'. Most unusual.
>
No idea. Perhaps that's what the comment on line 181 in
fcompiler/gnu.py was about?

>>
>> The above doesn't get rid of references to msvcrt.dll though. See
>> interpnd.txt and qhull.txt (output of Dependency Walker) in
>> https://sourceforge.net/projects/scipy/files/scipy/temp/. I am also
>> wondering if this dual runtime is really the issue here - it has
>> worked fine with this setup for a long time, and the segfault occurs
>> for recently added code (interpnd). Perhaps getting rid of free/malloc
>> is enough.
>
> The dual runtime is an issue in this case. Scipy.spatial.qhull uses
> msvcrt to allocate memory and scipy.interpolate.interpnd uses msvcrt90
> to free it.

Yes, that's clear. But I meant that we just need to resolve this
memory allocation issue, not the much broader issue of using two
runtimes at all - this happens in all .pyd files. But in most cases
only a few functions like _assert, _isatty, _tempnam are used from
msvcrt.dll. This is harmless as far as I can tell and probably can't
be avoided with MinGW.

> At this point it is probably best to use
> PyMem_Malloc/PyMem_Free, as Robert Kern suggested.

Agreed.

Ralf

>>
>> There is also an unclear warning at line 181 of fcompiler/gnu.py
>> (originally written by Pearu):
>>      # the following code is not needed (read: breaks) when using MinGW
>>      # in case want to link F77 compiled code with MSVC
>> I'm not sure if that just refers to the line "opt.append('gcc')" and
>> not also to the runtime_lib that you propose to change now.