[SciPy-Dev] Release blocker?

[Ralf Gommers]

On Sat, Feb 19, 2011 at 5:44 PM, Christoph Gohlke <cgohlke at uci.edu> wrote:
>
>
> On 2/19/2011 1:34 AM, Ralf Gommers wrote:
>> On Sat, Feb 19, 2011 at 5:09 PM, Christoph Gohlke<cgohlke at uci.edu>  wrote:
>>>
>>>
>>> On 2/19/2011 12:22 AM, Ralf Gommers wrote:
>>>> On Thu, Feb 17, 2011 at 8:39 AM, Robert Kern<robert.kern at gmail.com>    wrote:
>>>>> On Wed, Feb 16, 2011 at 18:25, Pauli Virtanen<pav at iki.fi>    wrote:
>>>>>> On Wed, 16 Feb 2011 17:53:45 -0600, Robert Kern wrote:
>>>>>
>>>>>>> That said, distutils should already be passing the right flags to mingw
>>>>>>> to link against msvcr90.dll unless if we are interfering. Can you show
>>>>>>> us the command lines that are being executed on your machine? I believe
>>>>>>> that using the environment variable DISTUTILS_DEBUG=1 will cause
>>>>>>> setup.py to print them out as it goes along.
>>>>>>
>>>>>> Build log is here: http://pav.iki.fi/tmp/build.log
>>>>>> It's a simple Numpy 1.5.1 + current mingw32 + python.org 3.1 setup.
>>>>>>
>>>>>> Apparently, what happens is that modules linked with g++ get -lmsvcr90,
>>>>>> whereas modules linked with gfortran (due to BLAS, apparently) don't get
>>>>>> it.
>>>>>
>>>>> Ah, that makes sense. numpy.distutils owns the link flags for Fortran
>>>>> modules. I think the problem is in fcompiler/gnu.py:get_libraries()
>>>>> (both of them). We don't add the right runtime library unless if the
>>>>> configured C compiler is 'msvc'. I think we need to add it
>>>>> unconditionally.
>>>>>
>>>> I tried that, https://github.com/rgommers/numpy/tree/mingw-runtime-1.5.x.
>>>> Then building scipy against this patched numpy 1.5.1 shows that also
>>>> g77 receives '-lmsvcr90'. Build log at http://pastebin.com/Zu4PB4ss.
>>>> The tests all pass for me under Wine. I don't have access to a Win7
>>>> system though, so can someone who does try the installer in
>>>> http://sourceforge.net/projects/scipy/files/scipy/temp/?
>>>
>>> Just tried it on Windows 7. Qhull.pyd fails to load with "ImportError:
>>> DLL load failed". Process Monitor reports it is failing to find/load a
>>> file named '<stdin>'. Most unusual.
>>>
>> No idea. Perhaps that's what the comment on line 181 in
>> fcompiler/gnu.py was about?
>>
>>>>
>>>> The above doesn't get rid of references to msvcrt.dll though. See
>>>> interpnd.txt and qhull.txt (output of Dependency Walker) in
>>>> https://sourceforge.net/projects/scipy/files/scipy/temp/. I am also
>>>> wondering if this dual runtime is really the issue here - it has
>>>> worked fine with this setup for a long time, and the segfault occurs
>>>> for recently added code (interpnd). Perhaps getting rid of free/malloc
>>>> is enough.
>>>
>>> The dual runtime is an issue in this case. Scipy.spatial.qhull uses
>>> msvcrt to allocate memory and scipy.interpolate.interpnd uses msvcrt90
>>> to free it.
>>
>> Yes, that's clear. But I meant that we just need to resolve this
>> memory allocation issue, not the much broader issue of using two
>> runtimes at all - this happens in all .pyd files.
>
> I agree.
>
>>But in most cases
>> only a few functions like _assert, _isatty, _tempnam are used from
>> msvcrt.dll. This is harmless as far as I can tell and probably can't
>> be avoided with MinGW.
>
> The remaining dependencies on msvcrt probably come from linking to
> libmoldname.a instead of libmoldname90.a. Pygame went through some
> trouble to remove all msvcrt dependencies
> <http://www.pygame.org/wiki/PreparingMinGW>.

I don't like that approach for two reasons:
1. It's manual tweaking that makes the build environment harder to
reproduce on other machines.
2. Pygame now links to a specific library, msvcr71.dll. For us this
could be msvcr90.dll. But what if the next version of Python is built
with Visual Studio 2010? Then you want msvcr100.dll, so you end up
with two runtimes again.

In my opinion we should only follow the Pygame example if there's a
really good reason to do so.

Ralf