[Python-Dev] Active Objects in Python

[Bruce Eckel]

According to this list's welcome message, I should introduce myself.
I'm Bruce Eckel, I write, consult and give seminars about computer
programming, I use Python whenever I can get away with it, and I've
spoken at Pycon a number of times. There are further URLs in my
signature at the bottom.

I'm joining this list because I was having a conversation with Guido
about concurrency support in Python (in particular, using an
actor/active-object approach) and he suggested I bring it over
here. Here are the highlights of my most recent reply to him (Guido's
remarks are marked by the '>' signs):

> Agreed. IMO the pthreads-style solution doesn't work well no matter
> *what* the programming model.

Exactly. I think the problem may be that low-level threads people are
one step behind (much like embedded systems programmers, which is
where I began). They may be just now catching up to objects, but as
far as concurrency goes their brains still think in terms of threads,
so it seems natural to apply thread concepts to objects.

But from an OO standpoint, pthread-style thinking is two steps
backwards. You effectively throw open the innards of the object that
you just spent time decoupling from the rest of your system, and the
coupling is now unpredictable. It's the worst of all possible worlds.

> I worked on Mobile Agents at CNRI from 1995 till 2000, but they were
> very heavy-weight. Yet, I think that any solution that moves objects
> between CPUs on a regular basis is going to suffer the same problem
> that process switching in general gets you -- the move is so expensive
> that it's hard to decide when it's justified.

The examples I've seen haven't relied on mobility, so that's a
question: how important is mobility really to an agent system? And
could it be done in a trivial fashion in Python, by sending source
code. And if it was really necessary sometimes, could it be something
where the cost of mobility only occurred when moving an object?

It's possible that either an argument can be made against mobility
and/or some kind of trivial mobility system could be developed that
would work when necessary. Possibly a Linda-like put-take system with
pickles (off the top of my head).

I don't know that much about mobility, but it does seem like mobile
agents could be a powerful solution to the problem solved by
enterprise messaging systems.

> I believe you pointed me to Active Objects before, right? (Years ago,
> maybe when you visited us at Zope.)

I may have mentioned them in the past.

> If multiple active objects can co-exist in the same *process*, but
> nevertheless the language implementation prevents them from sharing
> data except via channels, and in addition allows dynamic reallocation
> of active objects across multiple CPUs, they just might be the ticket.
> But it would require a complete implementation to prove it.

Yes, defining an class as "active" would:
1) Install a worker thread and concurrent queue in each object of that
class.
2) Automatically turn method calls into tasks and enqueue them
3) Prevent any other interaction other than enqueued messages

At that point, the only time the GIL might be needed is to lock the
queue at the point of putting ant taking objects. But there has been
work done on lock-free data structures, which has been incorporated
into Java 5 libraries, so it might even be possible to use a lock-free
queue to do this and thus eliminate the need for the GIL at all.

Since the enqueuing process serializes all requests to active objects,
and since each message-task runs to completion before the next one
starts, the problem of threads interfering with each other is
eliminated. Also, the enqueuing process will always happen, so even if
the queue blocks it will be for a very short, deterministic time. So
the theory is that Active Object systems cannot deadlock (although I
believe they can livelock).

So yes, the best way for this to work might be some kind of enforced
implementation -- but forcing you to do something is not particularly
Pythonic, so I would think that it would be possible to build an
active object framework along with some checking tools to warn you if
you are breaking the rules. But what's not clear is whether this would
require knowledge of the innards of the Python interpreter (which I
don't have) or if it could be built using libraries.

BTW: I think that Hoare's Communicating Sequential Processes (CSP)
basically describes the idea of active objects but without objects.
That is, I think active objects is CSP applied to OO.

Bruce Eckel    http://www.BruceEckel.com
Contains electronic books: "Thinking in Java 3e" & "Thinking in C++ 2e"
Web log: http://www.artima.com/weblogs/index.jsp?blogger=beckel