SWIG overhead

[Chris Mellon]

On 2/1/07, Diez B. Roggisch <deets at nospam.web.de> wrote:
> Bart Ogryczak wrote:
>
> > On Feb 1, 12:48 pm, "Diez B. Roggisch" <d... at nospam.web.de> wrote:
> >> > Yeah, found that one googling around. But I haven´t fund anything more
> >> > up to date. I imagine, that the performance of all of these wrappers
> >> > has been improved since then. But the performance of Python/C API
> >> > would too?
> >> > Anyways, it´s not about exact number, it´s more about taking decision
> >> > if doing rewrite is worth it´s time.
> >>
> >> The wrappers essentially create the boilerplate-code that invokes the
> >> Python C-API. So whatever improvements the latter has been developed, the
> >> wrappers will benefit from it.
> >
> > Without doubt it´s true in case of SWIG, but if I understand
> > Python.Boost documentation correctly, it does *not* use Python/C API.
>
> It has to. In the end, marshalling data between e.g. python datastructures
> and C++ is done that way.
>

As I understand it, part of the Boost.Python internals is a C++
wrapper over the Python C api, and there's no separate code generation
phase because it uses template magic to generate the wrappers. So
while obviously the C API is used at some level, it's not visible to
the wrapper author.

> >> I doubt that there are major performance penalties associated with any of
> >> them.
> >
> > Take a look at pages 23 and 24 of http://people.web.psi.ch/geus/talks/
> > europython2004_geus.pdf
>
> Ok, I see. I really wonder what SWIG does.
>

SWIG generates a low-level C module, and then a Python one on top of
it. It allows for quite some versatility in wrapping (because you can
have (almost) arbitrary Python code generated). Most of the other
tools generate a .pyd which you import directly.

> > Well, SWIG is easy to use. But I´ve gotta make hundreds of millions of
> > calls, which do tasks as simple, as getting one int from an array and
> > returning it. With functions that simple SWIG´s overhead seems to be a
> > problem.
>
> Still I think you should first use wrappers for ease of use. Then when you
> hit a performance bottleneck, it might be worth  wrapping that class
> manually. However, it _might_ of course be that this isn't integrating too
> seamless with the wrapped classes, but I can't say anything about that
> really.
>

Personally, I'd favor *correctness* first, then ease of use, and then speed.

> Diez
> --
> http://mail.python.org/mailman/listinfo/python-list