A matter of queues, tasks and multiprocessing

[John Nagle]

On 11/10/2010 9:25 AM, Emanuele D'Arrigo wrote:
> Greetings everybody,
>
> I've tried to come up with this message for a couple of weeks now and
> it doesn't look like I'm getting any clearer in my thoughts so I
> decided that it's probably best to take the plunge and ask you guys to
> kindly throw me a rope...
>
> What I'm trying to come up with is some kind of simple, dynamically
> scalable, domain-agnostic, multi-processor capable, tasking system.
> Maybe I'm already giving myself too many constraints here, i.e. it
> might all be possible except... the "simple" part.

    No, the problem is vagueness.  We can't tell from your description
what you want to do.  A real-time desktop application? Warehouse-scale
computing?  Your own "cloud"?

    Incidentally, bear in mind that once you scale beyond tens of
machines, error detection and recovery starts to dominate
the problem.

				John Nagle