2.6, 3.0, and truly independent intepreters

[Andy O'Meara]

Grrr... I posted a ton of lengthy replies to you and other recent
posts here using Google and none of them made it, argh. Poof. There's
nothing that fires more up more than lost work,  so I'll have to
revert short and simple answers for the time being.  Argh, damn.


On Oct 25, 1:26 am, greg <g... at cosc.canterbury.ac.nz> wrote:
> Andy O'Meara wrote:
> > I would definitely agree if there was a context (i.e. environment)
> > object passed around then perhaps we'd have the best of all worlds.
>
> Moreover, I think this is probably the *only* way that
> totally independent interpreters could be realized.
>
> Converting the whole C API to use this strategy would be
> a very big project. Also, on the face of it, it seems like
> it would render all existing C extension code obsolete,
> although it might be possible to do something clever with
> macros to create a compatibility layer.
>
> Another thing to consider is that passing all these extra
> pointers around everywhere is bound to have some effect
> on performance.


I'm with you on all counts, so no disagreement there.  On the "passing
a ptr everywhere" issue, perhaps one idea is that all objects could
have an additional field that would point back to their parent context
(ie. their interpreter).  So the only prototypes that would have to be
modified to contain the context ptr would be the ones that don't
inherently operate on objects (e.g. importing a module).


On Oct 25, 1:54 am, greg <g... at cosc.canterbury.ac.nz> wrote:
> Andy O'Meara wrote:
> > - each worker thread makes its own interpreter, pops scripts off a
> > work queue, and manages exporting (and then importing) result data to
> > other parts of the app.
>
> I hope you realize that starting up one of these interpreters
> is going to be fairly expensive. It will have to create its
> own versions of all the builtin constants and type objects,
> and import its own copy of all the modules it uses.
>

Yeah, for sure. And I'd say that's a pretty well established
convention already out there for any industry package.  The pattern
I'd expect to see is where the app starts worker threads, starts
interpreters in one or more of each, and throws jobs to different ones
(and the interpreter would persist to move on to subsequent jobs).

> One wonders if it wouldn't be cheaper just to fork the
> process. Shared memory can be used to transfer large lumps
> of data if needed.
>

As I mentioned, wen you're talking about intricate data structures, OS
opaque objects (ie. that have their own internal allocators), or huge
data sets, even a shared memory region unfortunately  can't fit the
bill.


Andy