[C++-sig] Iterating over small objects

[Jim Bosch]

On 01/19/2012 02:08 PM, babak wrote:
> Hi guys,
>
> Has anyone encountered (what feels like) quite slow iteration when looping
> over small containers ? I'm in a situation where I'm iterating over 100000+
> 3d vectors and its taking much longer than I'd expect. The vector type is
> bound via boost::python and has an iter method which uses range(being, end)
> for the iteration.
>
> I've attached a simple example binding and python test to illustrate what I
> mean more clearly. I'm under the impression that in a situation like this
> the overhead of creating the iterator is greater than the actual iteration,
> but thats just a guess. Alternatively my perspective on what the iteration
> speed should be might be completely skewed, so any help in clarifying what's
> going on here would be greatly appreciated.
>

My sense is that this is something that really can't be expected to be 
efficient across the C++/Python boundary.  There is some overhead 
associated with creating the exposures, and there's also some overhead 
in converting the types, especially if those vectors aren't builtin 
numeric types.

If you can redesign the C++/Python boundary to convert your vectors 
directly into Python lists or tuples instead of wrapped C++ vectors, I 
expect you'll see an improvement, because you'll be able to take 
advantage of whatever optimization the Python developers have put into 
iteration over those classes.  Of course, it's best by far to do 
whatever iteration you can in C++ if performance is important, but I'm 
guessing that's not an option here.

Jim