weave in 64 bit strange behavior

Tue May 14 20:44:10 EDT 2013

Hi everyone,

I realize my previous post was quite unreadable, thanks to my email
client. I am going to report my question here, with slight enhancements.
Apologies for inconvenience caused and spamming your mailboxes.

I am facing a strange problem using weave on 64 bit machine.
Specifically with weave's inline function. It has something to do with
weave's catalog. 

Similar issues I found in the past (very old)

http://mail.scipy.org/pipermail/scipy-dev/2006-June/005908.html 

http://mail.scipy.org/pipermail/scipy-dev/2005-June/003042.html

Common things I have in my observation are:

-          Already working setup in 32 bit doesn't work in same manner
in 64 bit env

-          Weave recompiles inline code which does not require any
recompilation. This is random behavior.  Whenever weave recompiles I see
a notification "repairing catalog by removing key" in the output which
ends up in the error message "ImportError: DLL load failed: Invalid
access to memory location"

-          Sometimes gcc gets into an infinite loop printing error
message "Looking for python27.dll". Even though the dll is on the path.
This process doesn't end. Had to kill it forcefully. G++ process became
ghost even after killing python process. 

Could someone advise what am I missing here. Is there any specific setup
that I need to do? Is there an issue with python 27 64 bit weave
implementation? 

Regards,

Alok

<CODE>

I have a simple script to calculate moving average using weave's inline
function. 

source mvg.py

import numpy as np

import scipy.weave as weave

import distutils.sysconfig

import distutils.dir_util

import os

distutils.sysconfig._config_vars["LDSHARED"]="-LC:\strawberry\c\x86_64-w
64-mingw32\lib"

def ExpMovAvg(data,time,lag):

    if (data.size!=time.size):

        print "error in EMA, data and time have different size"

        return None

    result=np.repeat(0.0,data.size)

    code="""

    #line 66 "basics.py"

    result(0)=data(0);

    for (int i=0;i<data.size()-1;i++)

    {

        //double alpha=1-(2.0/(lag+1)*(time(i)-time(i-1)));

        double dt=time(i+1)-time(i);

        double alpha=pow(0.13533, dt / lag);

        if(alpha>1)

        {

            alpha=10;

        }

        result(i+1)=(1-alpha)*data(i)+alpha*result(i);

    }

    """

weave.inline(code,["data","time","lag","result"],type_converters=weave.c
onverters.blitz,headers=["<math.h>"],compiler="gcc",verbose=2)

    return result

source test.py

import string

import numpy as np

import mvg

print(mvg.ExpMovAvg(np.array(range(10)),np.array(range(10)),2))

Output:

Working output:

Y:\STMM\alpha\klse\PROD>c:\python27\python.exe

s:\common\tools\python\python-2.7-64bit\test.py

[ 0.          0.          0.63212774  1.49679774  2.44701359  3.42869938

4.42196209  5.41948363  6.41857187  7.41823646]

Now if I keep running the script multiple times, sometimes I see correct

output... but suddenly sometimes I get below error.

Y:\STMM\alpha\klse\PROD>c:\python27\python.exe
s:\common\tools\python\python-2.7-64bit\test.py

repairing catalog by removing key

<weave: compiling>

Looking for python27.dll

running build_ext

running build_src

build_src

building extension "sc_44f3fe3c65d5c3feecb45d9269ac207f5" sources

build_src: building npy-pkg config files

Looking for python27.dll

customize Mingw32CCompiler

customize Mingw32CCompiler using build_ext

Looking for python27.dll

customize Mingw32CCompiler

customize Mingw32CCompiler using build_ext

building 'sc_44f3fe3c65d5c3feecb45d9269ac207f5' extension

compiling C++ sources

C compiler: g++ -g -DDEBUG -DMS_WIN64 -O0 -Wall

compile options: '-Ic:\python27\lib\site-packages\scipy\weave
-Ic:\python27\lib\site-packages\scipy\weave\scxx
-Ic:\python27\lib\site-packages\scipy\weave\blitz
-Ic:\python27\lib\site-packages\numpy\core\include -Ic:\python27\include
-Ic:\python27\PC -c' 

g++ -g -DDEBUG -DMS_WIN64 -O0 -Wall
-Ic:\python27\lib\site-packages\scipy\weave
-Ic:\python27\lib\site-packages\scipy\weave\scxx
-Ic:\python27\lib\site-packages\scipy\weave\blitz
-Ic:\python27\lib\site-packages\numpy\core\include -Ic:\python27\include
-Ic:\python27\PC -c
c:\users\ajadhav2\appdata\local\temp\ajadhav2\python27_compiled\sc_44f3f
e3c65d5c3feecb45d9269ac207f5.cpp -o
c:\users\ajadhav2\appdata\local\temp\ajadhav2\python27_intermediate\comp
iler_2d3e1e2e4de6a91419d2376b162e5342\Release\users\ajadhav2\appdata\loc
al\temp\ajadhav2\python27_compiled\sc_44f3fe3c65d5c3feecb45d9269ac207f5.
o

Found executable C:\strawberry\c\bin\g++.exe

g++ -g -DDEBUG -DMS_WIN64 -O0
-Wall-Ic:\python27\lib\site-packages\scipy\weave-Ic:\python27\lib\site-p
ackages\scipy\weave\scxx
-Ic:\python27\lib\site-packages\scipy\weave\blitz
-Ic:\python27\lib\site-packages\numpy\core\include-Ic:\python27\include
-Ic:\python27\PC -c
c:\python27\lib\site-packages\scipy\weave\scxx\weave_imp.cpp -o
c:\users\ajadhav2\appdata\local\temp\ajadhav2\python27_intermediate\comp
iler_2d3e1e2e4de6a91419d2376b162e5342\Release\python27\lib\site-packages
\scipy\weave\scxx\weave_imp.o

g++ -g -shared
c:\users\ajadhav2\appdata\local\temp\ajadhav2\python27_intermediate\comp
iler_2d3e1e2e4de6a91419d2376b162e5342\Release\users\ajadhav2\appdata\loc
al\temp\ajadhav2\python27_compiled\sc_44f3fe3c65d5c3feecb45d9269ac207f5.
o
c:\users\ajadhav2\appdata\local\temp\ajadhav2\python27_intermediate\comp
iler_2d3e1e2e4de6a91419d2376b162e5342\Release\python27\lib\site-packages
\scipy\weave\scxx\weave_imp.o -Lc:\python27\libs
-Lc:\python27\PCbuild\amd64 -lpython27 -lmsvcr90 -o
c:\users\ajadhav2\appdata\local\temp\ajadhav2\python27_compiled\sc_44f3f
e3c65d5c3feecb45d9269ac207f5.pyd

running scons

Traceback (most recent call last):

  File "s:\common\tools\python\python-2.7-64bit\test.py", line 5, in
<module>
print(mvg.ExpMovAvg(np.array(range(10)),np.array(range(10)),2))

  File "s:\common\tools\python\python-2.7-64bit\mvg.py", line 30, in
ExpMovAvgweave.inline(code,["data","time","lag","result"],
type_converters=weave.converters.blitz,headers=["<math.h>"],compiler="gc
c",verbose=2)

  File "c:\python27\lib\site-packages\scipy\weave\inline_tools.py",
line355, ininline **kw)

  File "c:\python27\lib\site-packages\scipy\weave\inline_tools.py", line
488, in compile_function exec 'import ' + module_name

  File "<string>", line 1, in <module> ImportError: DLL load failed:
Invalid access to memory location.

Y:\STMM\alpha\klse\PROD>

</CODE>

=============================================================================== 
Please access the attached hyperlink for an important electronic communications disclaimer: 
http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html 
=============================================================================== 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20130515/8f6bde02/attachment.html>