[SciPy-user] conversion of c++ vector in weave
Michael Sorich
mike_sorich at hotmail.com
Sun Oct 13 22:35:45 EDT 2002
Thanks for the help, Eric. Sorry about the lack of details.
The part that I want to speed up takes a distance matrix (array) and a
list of bins (which can be overlapping) and returns all the combinations
of bins that the values in the distance matrix upper triangle fall
within, as a list of bin index lists.
Example:
distMatrix = [[0.,1.,2.],[1.,0.,3.],[2.,3.,0.]]
bins = [[0.9,2.1],[1.9,3.1],[2.9,4.1]], where the inner lists contain
the min and max values of the bin.
In this example there are 3 distances to be binned (1.,2.,3.). distance
1 can be binned into bin 0, dist 2 into bins 0 and 1 and dist 3 into
bins 1 and 2. The combinations include [0,0,1],[0,0,2],[0,1,1],[0,1,2]
It is a little more complicated than above, in that the function takes
multiple dist matrices (these are stored in the third dimension of the
distMatrix variable).
I have implemented this in python using lists to store the combinations,
because I could not see any way to determine the size of the array at
the time of construction. I figured that vectors in C++ would be the
best replacement for the lists, however I was unsure what would be the
most efficient method to return the results stored in the vector. Thanks
for the examples of how to do this. What I have ended up doing is
creating a PyArray (since at the end the size of the vector is known)
and copying the data across and then returning the PyArray. It wasn't as
complicated as I had feared.
I have attached the python and c versions of the function, if you are
interested. The algorithm I made is a quick hack. If you know of a
better way please let me know!
Thanks again,
Michael
-----Original Message-----
From: scipy-user-admin at scipy.net [mailto:scipy-user-admin at scipy.net] On
Behalf Of eric jones
Sent: Friday, 11 October 2002 5:49 PM
To: scipy-user at scipy.net
Subject: RE: [SciPy-user] conversion of c++ vector in weave
Hey Michael,
Can you give me a little more information about your application --
especially what your data structure looks like. This'll help us
give more informed ideas on how to speed up the code and whether
weave is going to help or not.
If you are working with a list of numbers, then the following example
might give you some clues. Note, though, that the fastest versions
assume
you are using a homogeneous list of integers and doesn't do any error
checking at all. This'll lead to bad things if you pass in a list with
a
string in it. Further, if your data looks like this, your data would
fit
nicely into Numeric arrays, so I'm betting this isn't the ticket.
YOU'LL NEED THE LATEST WEAVE FROM CVS TO RUN THE EXAMPLES !!
I've made quite a few changes in the last week. I'm finally to the
point
of updating the documentation. When that is done, I'll release weave
0.3.
regards,
eric
#-----------------------------------------------------------------------
-----
# Example converting list to std::vector
# Typical run:
# h:\tmp>python std_vec.py
# a initially: [1, 2, 3]
# a on return: [101, 102, 103]
# time it: N = 1000000
# python-loop (sec): 0.56299996376
# python-list-comprehension (sec): 0.827999949455
# simple weave (sec): 0.139999985695
# fast weave(sec): 0.0470000505447
# weave-no-vector(sec): 0.0460000038147
#-----------------------------------------------------------------------
-----
import time
import weave
code = """
const int Na = a.length();
int i;
// convert to std::vector
std::vector<int> v_a(Na);
for(i = 0; i < Na; i++)
v_a[i] = a[i];
// do something with the vector
for(i = 0; i < Na; i++)
v_a[i] = v_a[i] + 100;
// put results back into a
for(i = 0; i < Na; i++)
a[i] = v_a[i];
"""
a = [1,2,3]
print "a initially:", a
weave.inline(code,['a'],headers=["<vector>"])
print "a on return:", a
N = 1000000
print "time it: N =", N
#-----------------------------------------------------------------------
-----
# Python
#-----------------------------------------------------------------------
-----
a = range(N)
t1 = time.time()
for i in xrange(N):
a[i] = a[i] + 100
t2 = time.time()
print "python-loop (sec):", t2 - t1
#-----------------------------------------------------------------------
-----
# Python List Comprehension
#-----------------------------------------------------------------------
-----
a = range(N)
t1 = time.time()
a = [x + 100 for x in a]
t2 = time.time()
print "python-list-comprehension (sec):", t2 - t1
#-----------------------------------------------------------------------
-----
# Simple weave
#-----------------------------------------------------------------------
-----
a = range(N)
t1 = time.time()
weave.inline(code,['a'],headers=["<vector>"])
t2 = time.time()
print "simple weave (sec):", t2 - t1
#-----------------------------------------------------------------------
-----
# Python API
#-----------------------------------------------------------------------
-----
api_code = """
const int Na = a.length();
int i;
// convert to std::vector
std::vector<int> v_a(Na);
for(i = 0; i < Na; i++)
v_a[i] = PyInt_AS_LONG(PyList_GET_ITEM(py_a,i));
// do something with the vector
for(i = 0; i < Na; i++)
v_a[i] = v_a[i] + 100;
// put results back into a
for(i = 0; i < Na; i++)
PyList_SET_ITEM(py_a,i,PyInt_FromLong(v_a[i]));
"""
a = [1,2,3]
weave.inline(api_code,['a'],headers=["<vector>"])
a = range(N)
t1 = time.time()
weave.inline(api_code,['a'],headers=["<vector>"])
t2 = time.time()
print "fast weave(sec):", t2 - t1
#-----------------------------------------------------------------------
-----
# Python API -- no std::vector
#-----------------------------------------------------------------------
-----
api_code = """
const int Na = a.length();
int i, val;
// convert to std::vector
for(i = 0; i < Na; i++)
{
val = PyInt_AS_LONG(PyList_GET_ITEM(py_a,i));
PyList_SET_ITEM(py_a,i,PyInt_FromLong(val+100));
}
"""
a = [1,2,3]
weave.inline(api_code,['a'],headers=["<vector>"])
a = range(N)
t1 = time.time()
weave.inline(api_code,['a'],headers=["<vector>"])
t2 = time.time()
print "weave-no-vector(sec):", t2 - t1
> -----Original Message-----
> From: scipy-user-admin at scipy.net [mailto:scipy-user-admin at scipy.net]
On
> Behalf Of Michael Sorich
> Sent: Friday, October 11, 2002 2:02 AM
> To: scipy-user at scipy.net
> Subject: [SciPy-user] conversion of c++ vector in weave
>
> Hi
>
> I wish to convert a small part of my code to C++ for speed. The code
is
> not amenable to numeric arrays and uses lists extensively. I am
thinking
> about using c++ vectors in place of python lists (I presume that this
> will be a lot faster than using Py::List). What little experience I
have
> in extending python with C/C++ is limited to using weave.
>
> If I use vectors, is there any method to automatically convert my
result
> (vector< vector<int> >) to something that can be returned to python
> (tuple/list/array). I believe that Boost can do this (at least the
> upcoming v2, see
>
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/*checkout*/boost/boost/li
> bs/python/doc/v2/faq.html), however I am not sure how I would be able
to
> integrate this with weave.inline. It does not appear obvious that
there
> is anything in CXX that will do this.
>
> Any help would be appreciated.
>
> Thanks
>
> Michael Sorich
> PhD Student
> School of Pharmaceutical, Molecular and Biomedical Sciences
> University of South Australia
> Email: michael.sorich at postgrads.unisa.edu.au
> mike_sorich at hotmail.com
>
>
>
> ---
> Outgoing mail is certified Virus Free.
> Checked by AVG anti-virus system (http://www.grisoft.com).
> Version: 6.0.393 / Virus Database: 223 - Release Date: 30/09/2002
>
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.net
> http://www.scipy.net/mailman/listinfo/scipy-user
_______________________________________________
SciPy-user mailing list
SciPy-user at scipy.net
http://www.scipy.net/mailman/listinfo/scipy-user
---
Incoming mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.393 / Virus Database: 223 - Release Date: 30/09/2002
---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.393 / Virus Database: 223 - Release Date: 30/09/2002
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: testBinMap.py
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20021014/96a3b3ec/attachment.ksh>
More information about the SciPy-User
mailing list