Efficient data flow between orthogonal C extensions?

[Bernhard Herzog]

Robin Becker <robin at jessikat.fsnet.co.uk> writes:

> >mlh at furu.idi.ntnu.no (Magnus Lie Hetland) writes:
> >
> >> I'm about to write a component-based data flow program, and I'm
> >> pondering the design, partly when it comes to efficiency. Basically,
> >> the system will instantiate a set of objects, set up in a directed
> >> network-like structure, and they will pass data among themselves in a
> >> stream-like fashion.
[...]
> I believe Bernhard Herzog's Sketch drawing editor does stream like
> transfers between extensions.

Indeed. It's something similar to postscript's filters, that is file
like objects that transform data on the fly. The stream filters are in
the Filters subdirectory in Sketch's sources. One place where it's used
in another C module outside of the filter implementation itself in
Sketch/Modules/pstokenize.c .

I'm not sure how similar it is to what Magnus wants to do, but at least
the C-API handling might be an example.

The basic approach taken to communicate a C-API from one dynamically
loaded module to another is that the streamfilter module as an attribute
called Filter_Functions which holds a PyCObject containing a pointer to
a struct with function pointers. The pstokenize module imports the
streamfilter module, extracts the struct and then calls the functions.

The approach is basically the same as in cStringIO in the Python sources
(see Include/cStringIO.h and Modules/cStringIO.c. PyGTK also does
somethng like this.

   Bernhard

-- 
Intevation GmbH                                 http://intevation.de/
Sketch                                 http://sketch.sourceforge.net/
MapIt!                                           http://www.mapit.de/