FW: [SciPy-user] Problem with Weave
Bob.Cowdery at CGI-Europe.com
Bob.Cowdery at CGI-Europe.com
Thu Jul 15 04:19:06 EDT 2004
Eric
Update on problem plus another question:
As I didn't get a response yet I thought I would add what else I have found
out. This is fairly bizarre. When I do a code update if I then do a fair
number of iterations running from the main in that module before I try to
run the application, most times the app then works. If I just do one
iteration to compile the code generally the app will still fail. If I do a
code update and run the app straight away it always fails. I also notice
that I sometimes see a message saying the catalogue is corrupt and being
removed but I haven't tied down exactly when this occurs. When it fails the
symptoms are as the posting below.
The good news is the code flies. I have gone from 100% processor and a time
that well overshot my 46ms timebox to a few % utilisation <1ms execution
time. The way variables are just punted in and out of the 'C' code
automatically is also great.
Question: I pass in a lot of arrays which need to persist across calls and
that works great. I also pass in two scalars that also need to persist but
return_val presumable can only return one value as per normal with 'C'. What
I have done to get round this is pass in a pre-dimensioned array and I copy
my two values into this at the end of the 'C' code then pull them back out
into class variables in the python code. Is this the best way to do this?
Thanks
Bob
-----Original Message-----
From: Cowdery, Bob [UK]
Sent: 12 July 2004 12:55
To: 'SciPy Users List'
Subject: RE: [SciPy-user] Problem with Weave
Hi Eric
Thanks for the pointers. Yes that bit works fine now.
----
Unfortunately I have run into a much more obtuse problem. I converted a
piece of code that had a nested loop as I could only get rid of the inner
loop with array functions and it was still too slow by a large factor.
However I don't think the actual code is causing this problem as almost any
code blows up on me.
The code is at the end just to complete the story. This code compiles and
tests out ok if I run it directly from a main within the same module. When I
try to run it within the application, it compiles ok and I get 'file
changed' then every time it is invoked I get an exception:
Exception exceptions.RuntimeError: 'maximum recursion depth exceeded' in
<bound method module.__del__ of <win32 module '__eq__' (0 functions)>>
ignored
The function is not called recursively, however it is called on a separate
thread every 46ms or so. I thought at first it might be getting called
mutiple time while it was compiling but it's synchronous so the thread is
forced to wait first time while it compiles. After this failure it locks up
the application. Once the application exits I am left with
gnuplot_helper.exe and wgnuplot_helper.exe (neither of which I use) taking
all my processer and I have to kill them off. Before that I just notice
dumpreg is there and then it exits.
It's very unpredictable, for example first thing this morning I ran it and
it appeared to be running through although it wasn't actually doing what it
should. As I wasn't sure if it had really compiled the current text I
replaced the code with a printf() and it failed. Since then I can't get it
to run through at all EVEN WITH no code in the 'code' variable. I even
removed the array assignments and all the parameters incase they were the
problem. I couldn't figure out how to pass in class variables directly,
that's why they are all copied to local variables.
I am completely bemused by what's going on here. Any ideas?
Bob.
------------
code = """
double ISum = 0.0;
double QSum = 0.0;
double lsb = 0.0;
double usb = 0.0;
for(int smpl = 0 ; smpl < IN_SZ ; smpl++)
{
I_delay[delay_ptr] = I_in[smpl];
Q_delay[delay_ptr] = Q_in[smpl];
if(++delay_ptr == DELAY_LEN) delay_ptr = 0;
ISum = 0.0;
QSum = 0.0;
for(int tap = -40 ; tap <= 40 ; tap++)
{
ISum = ISum + filter_phasing[tap+(DELAY_LEN-1)/2] *
I_delay[delay_ptr];
QSum = QSum + filter_phasing[(tap-(DELAY_LEN-1)/2)-1] *
Q_delay[delay_ptr];
if(++delay_ptr == DELAY_LEN) delay_ptr = 0;
}
if(peak>1) peak = peak * agcdecay;
if(sideband == 0)
{
lsb = ISum + QSum;
if(abs(lsb) > peak) peak = abs(lsb);
real_out[smpl] = int(lsb*16384.0/peak);
}
else
{
usb = ISum - QSum;
if(abs(usb) > peak) peak = abs(usb);
real_out[smpl] = int(usb*16384.0/peak);
}
}
"""
IN_SZ = len(self.m_I_in)
I_in = self.m_I_in
Q_in = self.m_Q_in
I_delay = self.m_I_Delay
Q_delay = self.m_Q_Delay
delay_ptr = self.m_delay_ptr
DELAY_LEN = self.DELAY_LEN
filter_phasing = self.m_filter_phasing
real_out = self.m_real_out
#weave.inline(code, ['sideband', 'agcdecay', 'peak', 'I_in',
'IN_SZ', 'Q_in', 'I_delay', 'Q_delay', 'delay_ptr', 'DELAY_LEN',
'filter_phasing', 'real_out'])
-----Original Message-----
From: eric jones [mailto:eric at enthought.com]
Sent: 12 July 2004 07:49
To: SciPy Users List
Subject: Re: [SciPy-user] Problem with Weave
Hey Bob,
The docs are seriously out of date. Sorry about that.
Much of the crud that you used to need for handling C++->Python type
conversions is now gone. Here is how I wrote your example:
C:\Documents and Settings\eric>python
Enthought Edition build 1057
Python 2.3.3 (#51, Feb 16 2004, 04:07:52) [MSC v.1200 32 bit (Intel)] on
win32
Type "help", "copyright", "credits" or "license" for more information. >>>
import weave >>> a=1 >>> a=weave.inline("return_val=a+1;",['a'])
file changed
msvc
No module named msvccompiler in scipy_distutils, trying from distutils.. No
module named msvccompiler in scipy_distutils, trying from distutils..
Missing compiler_cxx fix for MSVCCompiler >>> a 2
The output is quite verbose now (too verbose), but the amount of code
you need to write is less. The weave/examples files (weave will be
installed in your site-packages directory) generally have been updated
to the new syntax and will provide some help. Also, if you are a C++
person, then looking at the weave/scxx/object.h file will give some
hints as to what is going on.
see ya,
eric
Bob.Cowdery at CGI-Europe.com wrote:
> I am about to convert some python code to 'C' so am going through the
> Weave documentation. I can get most things to work except returning
> values from the 'C' code. I would guess the correct libraries are not
> getting into the compile but I don't know how to fix it. I can compile
> and run any 'C' code fine until I use any of the Py:: classes.
>
> As an example if I type:
>
>>>> import weave
>>>> a=1
>>>> a=weave.inline("return_val =
> Py::new_reference_to(Py::Int(a+1));",['a'], verbose=2)
>
> this gives me:
>
> file changed
> running build_ext
> building 'sc_5b09eaf68ff529a1fbaedc892ca5a4531' extension C:\Program
> Files\Microsoft Visual Studio .NET 2003\Vc7\bin\cl.exe /c /nologo /Ox
> /MD /W3 /GX -IC:\Python22\lib\site-packages\weave
> -IC:\Python22\lib\site-packages\weave\scxx -IC:\Python22\include
> /TpC:\DOCUME~1\cowderyb\LOCALS~1\Temp\CowderyB\python22_compiled\sc_5b
> 09eaf68ff529a1fbaedc892ca5a4531.cpp
>
/FoC:\DOCUME~1\cowderyb\LOCALS~1\Temp\CowderyB\python22_intermediate\compile
r_ba943cc959c75c0df25fd4c114ec382f\Release\sc_5b09eaf68ff529a1fbaedc892ca5a4
531.obj
> Traceback (most recent call last):
> File "<input>", line 1, in ?
> File "C:\Python22\lib\site-packages\weave\inline_tools.py", line
> 335, in inline
> auto_downcast = auto_downcast,
> File "C:\Python22\lib\site-packages\weave\inline_tools.py", line
> 439, in compile_function
> verbose=verbose, **kw)
> File "C:\Python22\lib\site-packages\weave\ext_tools.py", line 340,
> in compile
> verbose = verbose, **kw)
> File "C:\Python22\lib\site-packages\weave\build_tools.py", line 272,
> in build_extension
> setup(name = module_name, ext_modules = [ext],verbose=verb)
> File "C:\Python22\lib\site-packages\scipy_distutils\core.py", line
> 42, in setup
> return old_setup(**new_attr)
> File "C:\Python22\lib\distutils\core.py", line 157, in setup
> raise SystemExit, "error: " + str(msg)
> CompileError: error: command '"C:\Program Files\Microsoft Visual
> Studio .NET 2003\Vc7\bin\cl.exe"' failed with exit status 2
>
> Any help appreciated.
>
> Thanks
> Bob
>
>
> Bob Cowdery
> CGI Senior Technical Architect
> +44(0)1438 791517
> Mobile: +44(0)7771 532138
> bob.cowdery at cgi-europe.com <mailto:bob.cowdery at cgi-europe.com>
>
>
>
>
> **** Confidentiality Notice **** Proprietary/Confidential Information
> belonging to CGI Group Inc. and its affiliates may be contained in
> this message. If you are not a recipient indicated or intended in this
> message (or responsible for delivery of this message to such person),
> or you think for any reason that this message may have been addressed
> to you in error, you may not use or copy or deliver this message to
> anyone else. In such case, you should destroy this message and are
> asked to notify the sender by reply email.
>
>-----------------------------------------------------------------------
>-
>
>_______________________________________________
>SciPy-user mailing list
>SciPy-user at scipy.net http://www.scipy.net/mailman/listinfo/scipy-user
>
>
_______________________________________________
SciPy-user mailing list
SciPy-user at scipy.net http://www.scipy.net/mailman/listinfo/scipy-user
*** Confidentiality Notice *** Proprietary/Confidential
Information belonging to CGI Group Inc. and its affiliates
may be contained in this message. If you are not a recipient
indicated or intended in this message (or responsible for
delivery of this message to such person), or you think for
any reason that this message may have been addressed to you
in error, you may not use or copy or deliver this message
to anyone else. In such case, you should destroy this
message and are asked to notify the sender by reply email.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20040715/8de61ae4/attachment.html>
More information about the SciPy-User
mailing list